Introduction to Computer Vision: Plant Seedlings Classification

Problem Statement

Context

In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.

Objective

The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.

Data Dictionary

The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.

• The dataset can be download from Olympus.

• The data file names are:

  • images.npy
  • Labels.csv

• Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.

• The goal of the project is to create a classifier capable of determining a plant's species from an image.

List of Species

• Black-grass
• Charlock
• Cleavers
• Common Chickweed
• Common Wheat
• Fat Hen
• Loose Silky-bent
• Maize
• Scentless Mayweed
• Shepherds Purse
• Small-flowered Cranesbill
• Sugar beet

Note: Please use GPU runtime on Google Colab to execute the code faster.

Importing necessary libraries

# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==2.03 opencv-python==4.8.0.76 -q --user

#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 opencv-python==4.8.0.76 -q --user

# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user

#!pip install pandas==2.0.3 -q --user

#!pip install typing-extensions==4.6.0 -q --user

#!pip install scikeras==0.11.0 keras==2.15.0 scikit-learn==1.4.2 -q --user

Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.

import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import math
import cv2
import seaborn as sns

# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras import optimizers
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix,classification_report
from sklearn.preprocessing import LabelBinarizer

#Importing ImageGrid to plot the plant sample images
from mpl_toolkits.axes_grid1 import ImageGrid

from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random

# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

2024-07-12 16:04:19.866070: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

#Importing cv2_imshow for displaying images, but only if running on COLAB

try:
  from google.colab.patches import cv2_imshow
  IN_COLAB = True
except:
  IN_COLAB = False

SHARPEN = False #should we sharpen images
BLUR = False #should we blur images NOTE, we would only do one or the other, so could use 1 boolean
RESIZE = True #should we resize images

epochs = 25
batch_size = 32

print(SHARPEN)
print(RESIZE)
print(BLUR)
print(epochs)
print(batch_size)

False
True
False
25
32

print(IN_COLAB)

False

print(tf.__version__)
print(sns.__version__)
print(np.__version__)
print(pd.__version__)
print(cv2.__version__)
#opencv-python.__version__
print(np.__version__)

2.15.0
0.11.1
1.24.3
1.5.2
4.8.0
1.24.3

Loading the dataset

# Uncomment and run the below code if you are using google colab
if IN_COLAB:
    print("Running in Colab")
    from google.colab import drive
    drive.mount('/content/drive')

# Load the image file of dataset
# Load the labels file of dataset

if IN_COLAB:
    print("Running in Colab")
    images = np.load("/content/drive/MyDrive/Colab Notebooks/CV/images.npy")
    labels = pd.read_csv("/content/drive/MyDrive/Colab Notebooks/CV/Labels.csv")
else:
    images = np.load('images.npy')
    labels = pd.read_csv('labels.csv')

Data Overview

Understand the shape of the dataset

print(images.shape)
print(labels.shape)

(4750, 128, 128, 3)
(4750, 1)

Creating a list of category names

labels_plants = np.unique(labels)
print(labels_plants)

#store this for use elsewhere
num_labels = len(labels_plants)
print(num_labels)

['Black-grass' 'Charlock' 'Cleavers' 'Common Chickweed' 'Common wheat'
 'Fat Hen' 'Loose Silky-bent' 'Maize' 'Scentless Mayweed'
 'Shepherds Purse' 'Small-flowered Cranesbill' 'Sugar beet']
12

Observations:

• There are 12 groupings of image types
• A total of 4750 plant images
• Each image is size 128x128 pixels
• The number of channels is 3, images are RGB (Red, Green, Blue)

Exploratory Data Analysis

# switch to images copy so we can not have to reload if possible when we change things
images_copy = images.copy()

#Show a single sample image
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(images_copy[1000])
else:
    plt.imshow(images_copy[1000])

Observations

• We can observe that the images are being shown in BGR when plotted with openCV. We will need to convert to RGB for better viewing and recognition

#Show sample images
#defining a grid of size 12X12
fig = plt.figure(1, figsize=(num_labels, num_labels))
grid = ImageGrid(fig, 111, nrows_ncols=(num_labels, num_labels), axes_pad=0.05)
i = 0
index = labels.index

# 12 images from each category
for category_id, category in enumerate(labels_plants):
  condition = labels["Label"] == category
  plant_indices = index[condition].tolist()
  for j in range(0,num_labels):
      ax = grid[i]
      image_rgb = cv2.cvtColor(images_copy[plant_indices[j]], cv2.COLOR_BGR2RGB)                      # Adding subplots with 3 rows and 4 columns
      ax.imshow(image_rgb)
      ax.axis('off')
      if i % num_labels == num_labels - 1:
        #print the names for each category
        ax.text(200, 70, category, verticalalignment='center')
      i += 1
plt.show();

if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(images_copy[1000])
else:
    plt.imshow(cv2.cvtColor(images_copy[1000], cv2.COLOR_BGR2RGB))

plt.rcParams["figure.figsize"] = (12,5)
sns.countplot(x=labels.iloc[:,-1],order = labels['Label'].value_counts().index, palette='Spectral')
plt.xlabel('Plant Categories')
plt.xticks(rotation=45)

(array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11]),
 [Text(0, 0, 'Loose Silky-bent'),
  Text(1, 0, 'Common Chickweed'),
  Text(2, 0, 'Scentless Mayweed'),
  Text(3, 0, 'Small-flowered Cranesbill'),
  Text(4, 0, 'Fat Hen'),
  Text(5, 0, 'Charlock'),
  Text(6, 0, 'Sugar beet'),
  Text(7, 0, 'Cleavers'),
  Text(8, 0, 'Black-grass'),
  Text(9, 0, 'Shepherds Purse'),
  Text(10, 0, 'Common wheat'),
  Text(11, 0, 'Maize')])

Observations:

• This data set is imbalanced
• "Loose Silky-bent" plant images are the largest group, followed by "Common Chickweed" and "Scentless Mayweed"
• The smallest image groupings are "Sheperds Purse", "Common Wheat" and "Maize"

Data Pre-Processing

# switch paths to have a processed files
images_to_process = images_copy.copy() #leaving in place in case I decide to look at lap
images_to_resize = images_copy.copy()

Convert and/or resize the images based on flags set earlier

images_to_resize.shape

(4750, 128, 128, 3)

As we can see, the images are currently 128 x 128 x 3

# Resizing the image size to half ie., from 128X128 to 64X64 and converting to RGB
images_color_resized=[]
for i in range(len(images_to_resize)):
    
    #Apply sharpen filter
    if SHARPEN:
        kernel = np.array([[0, -1, 0],
                   [-1, 5,-1],
                   [0, -1, 0]])
        images_to_resize[i] = cv2.filter2D(src=images_to_resize[i], ddepth=-1, kernel=kernel)
    
    #Apply Gaussian blur
    if BLUR:
        images_to_resize[i] = cv2.GaussianBlur(images_to_resize[i],(5,5),0)

    #Convert the BGR images to RGB images
    images_to_resize[i] = cv2.cvtColor(images_to_resize[i], cv2.COLOR_BGR2RGB) 
    
    #Resize images
    if RESIZE:
        images_color_resized.append( cv2.resize(images_to_resize[i], None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)) 
    else:
        images_color_resized.append( cv2.resize(images_to_resize[i], None, fx=1.0, fy=1.0, interpolation=cv2.INTER_LINEAR))

np.shape(images_color_resized)

(4750, 64, 64, 3)

Observations

• We can see the images are now 64x64 in size.
• They retain the color channels.

Image before resizing

    
print('\n')
print("Image before resizing")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(images_copy[1000])
else:
    plt.imshow(cv2.cvtColor(images_copy[1000], cv2.COLOR_BGR2RGB))

Image before resizing

Observations

This image is pretty blurry already, suggesting it was resized once already. Resizing again may cause it to lose more data, affecting accuracy.

Image resized with no sharpening

image_resize = cv2.resize(images_copy[1000], None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)

print('\n')
print("Image resized with no sharpening/blurring")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(images_copy[1000])
else:
    plt.imshow(cv2.cvtColor(image_resize, cv2.COLOR_BGR2RGB))

Image resized with no sharpening/blurring

Observations

After resizing, this image becomes much less distinctive, suggesting resizing may affect accuracy.

Image after sharpening before resizing

kernel = np.array([[0, -1, 0],
                   [-1, 5,-1],
                   [0, -1, 0]])
image_sharp = cv2.filter2D(src=images_copy[1000], ddepth=-1, kernel=kernel)

print('\n')
print("Original image after sharpening")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(images_copy[1000])
else:
    plt.imshow(cv2.cvtColor(image_sharp, cv2.COLOR_BGR2RGB))

Original image after sharpening

Observations

Sharpening this image makes the edges appear a bit more distinctive and may help in identifying the plant.

image_blur = cv2.GaussianBlur(images_copy[1000],(5,5),0)

print('\n')
print("Original image after blurring")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(images_copy[1000])
else:
    plt.imshow(cv2.cvtColor(image_blur, cv2.COLOR_BGR2RGB))

Original image after blurring

Observations

Blurring this image makes the edges much softer. This may not lead to good identification accuracy.

Image after resizing

kernel = np.array([[0, -1, 0],
                   [-1, 5,-1],
                   [0, -1, 0]])
image_sharp = cv2.filter2D(src=images_copy[1000], ddepth=-1, kernel=kernel)
image_sharp_resize = cv2.resize(image_sharp, None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)

print('\n')
print("Original image after sharpening/resizing")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(image_sharp_resize)
else:
    plt.imshow(cv2.cvtColor(image_sharp_resize, cv2.COLOR_BGR2RGB))

Original image after sharpening/resizing

Observations

This image becomes much less distinctive once resized, even with sharpening.

image_blur = cv2.GaussianBlur(images_copy[1000],(5,5),0)
image_blur_resize = cv2.resize(image_blur, None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)

print('\n')
print("Original image after blurring/resizing")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(image_blur_resize)
else:
    plt.imshow(cv2.cvtColor(image_blur_resize, cv2.COLOR_BGR2RGB))

Original image after blurring/resizing

Observations

After blurring and resizing, this image become much softer at the edges.

Data Preparation for Modeling

• As we do not have many images in our dataset, we will only use 10% of our data for testing, 10% of our data for validation and 80% of our data for training.
• We will split the dataset into three parts, train,test and validation.

X_temp, X_test, y_temp, y_test = train_test_split(np.array(images_color_resized),labels , test_size=0.1, random_state=42,stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp,y_temp , test_size=0.1, random_state=42,stratify=y_temp)

print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)

(3847, 64, 64, 3) (3847, 1)
(428, 64, 64, 3) (428, 1)
(475, 64, 64, 3) (475, 1)

Encoding the target labels

# Convert labels from names to one hot vectors.

enc = LabelBinarizer()
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded=enc.transform(y_val)
y_test_encoded=enc.transform(y_test)

y_train_encoded.shape,y_val_encoded.shape,y_test_encoded.shape 

((3847, 12), (428, 12), (475, 12))

Data Normalization

Since the image pixel values range from 0-255, our method of normalization here will be scaling - we shall divide all the pixel values by 255 to standardize the images to have values between 0-1.

print('\n')
print("Image before normalization")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(X_train[1])
else:
    plt.imshow(X_train[1])

Image before normalization

# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

print('\n')
print("Image after normalization")
if IN_COLAB:
    print("Running in Colab")
    cv2_imshow(X_train_normalized[1])
else:
    plt.imshow(X_train_normalized[1])

Image after normalization

Observations

• Image appears unchanged after normalization

Model Building

Store model results in a dataframe for later comparison.

training_metrics = pd.DataFrame(columns=["accuracy"])
test_metrics = pd.DataFrame(columns=["accuracy"])

Build first model (basic) containing multiple layers for image processing and dense layer for classification

CNN Model layers:

• Convolutional input layer, 128 feature maps with a size of 3X3 and a * rectifier activation function
• Max Pool layer with size 2×2 and a stride of 2
• Convolutional layer, 64 feature maps with a size of 3X3 and a rectifier activation function.
• Max Pool layer with size 2×2 and a stride of 2
• Convolutional layer, 32 feature maps with a size of 3X3 and a rectifier activation function.
• Max Pool layer with size 2×2 and a stride of 2
• Flatten layer
• Fully connected or Dense layers (with 16 neurons) with Relu Act.
• Dropout layer to reduce overfitting or for regularization
• Output layer with Softmax to detect multiple categories (12).

# Clearing backend
backend.clear_session()

model_name = "Base Model"

# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# verify image sizes for input layer
np.shape(X_train_normalized)

(3847, 64, 64, 3)

# Since I will be trying various combos of resizing set the input layer dynamically
h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
base_model = Sequential()

# create convolution and max-pooling layers with input size of images
base_model.add(Conv2D(128, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
base_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
base_model.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
base_model.add(Flatten())

# add a fully connected layer
base_model.add(Dense(16, activation='relu'))
base_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
base_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
base_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
base_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 128)       3584      
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 128)       0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        73792     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 32)        18464     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 32)          0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 16)                32784     
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 128828 (503.23 KB)
Trainable params: 128828 (503.23 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

Fitting the model on the train data

base_history = base_model.fit(
            X_train_normalized, y_train_encoded,
            epochs=50,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=64,
            verbose=2
)

Epoch 1/50
61/61 - 9s - loss: 2.4596 - accuracy: 0.1003 - val_loss: 2.4518 - val_accuracy: 0.1379 - 9s/epoch - 146ms/step
Epoch 2/50
61/61 - 8s - loss: 2.4410 - accuracy: 0.1198 - val_loss: 2.4196 - val_accuracy: 0.1472 - 8s/epoch - 124ms/step
Epoch 3/50
61/61 - 8s - loss: 2.3448 - accuracy: 0.2028 - val_loss: 2.1197 - val_accuracy: 0.3458 - 8s/epoch - 133ms/step
Epoch 4/50
61/61 - 9s - loss: 2.1166 - accuracy: 0.3002 - val_loss: 1.9292 - val_accuracy: 0.3762 - 9s/epoch - 140ms/step
Epoch 5/50
61/61 - 9s - loss: 1.9635 - accuracy: 0.3273 - val_loss: 1.8374 - val_accuracy: 0.3949 - 9s/epoch - 147ms/step
Epoch 6/50
61/61 - 9s - loss: 1.8773 - accuracy: 0.3481 - val_loss: 1.6673 - val_accuracy: 0.4486 - 9s/epoch - 151ms/step
Epoch 7/50
61/61 - 9s - loss: 1.7978 - accuracy: 0.3587 - val_loss: 1.6041 - val_accuracy: 0.4813 - 9s/epoch - 152ms/step
Epoch 8/50
61/61 - 9s - loss: 1.6865 - accuracy: 0.3962 - val_loss: 1.5025 - val_accuracy: 0.5280 - 9s/epoch - 150ms/step
Epoch 9/50
61/61 - 10s - loss: 1.5977 - accuracy: 0.4292 - val_loss: 1.3621 - val_accuracy: 0.5514 - 10s/epoch - 158ms/step
Epoch 10/50
61/61 - 9s - loss: 1.4977 - accuracy: 0.4609 - val_loss: 1.3426 - val_accuracy: 0.5257 - 9s/epoch - 153ms/step
Epoch 11/50
61/61 - 10s - loss: 1.4603 - accuracy: 0.4762 - val_loss: 1.2123 - val_accuracy: 0.5935 - 10s/epoch - 158ms/step
Epoch 12/50
61/61 - 10s - loss: 1.3659 - accuracy: 0.5095 - val_loss: 1.1974 - val_accuracy: 0.6121 - 10s/epoch - 159ms/step
Epoch 13/50
61/61 - 9s - loss: 1.3370 - accuracy: 0.5272 - val_loss: 1.1135 - val_accuracy: 0.6449 - 9s/epoch - 153ms/step
Epoch 14/50
61/61 - 9s - loss: 1.2713 - accuracy: 0.5428 - val_loss: 1.1686 - val_accuracy: 0.6121 - 9s/epoch - 153ms/step
Epoch 15/50
61/61 - 10s - loss: 1.2636 - accuracy: 0.5425 - val_loss: 1.0778 - val_accuracy: 0.6636 - 10s/epoch - 165ms/step
Epoch 16/50
61/61 - 10s - loss: 1.2194 - accuracy: 0.5586 - val_loss: 1.0727 - val_accuracy: 0.6612 - 10s/epoch - 163ms/step
Epoch 17/50
61/61 - 10s - loss: 1.2052 - accuracy: 0.5620 - val_loss: 1.0273 - val_accuracy: 0.6589 - 10s/epoch - 167ms/step
Epoch 18/50
61/61 - 10s - loss: 1.1831 - accuracy: 0.5677 - val_loss: 1.0013 - val_accuracy: 0.6682 - 10s/epoch - 168ms/step
Epoch 19/50
61/61 - 10s - loss: 1.1410 - accuracy: 0.5745 - val_loss: 1.0001 - val_accuracy: 0.6799 - 10s/epoch - 171ms/step
Epoch 20/50
61/61 - 10s - loss: 1.1024 - accuracy: 0.5820 - val_loss: 0.9770 - val_accuracy: 0.6893 - 10s/epoch - 161ms/step
Epoch 21/50
61/61 - 10s - loss: 1.1224 - accuracy: 0.5854 - val_loss: 1.0176 - val_accuracy: 0.6893 - 10s/epoch - 161ms/step
Epoch 22/50
61/61 - 9s - loss: 1.0815 - accuracy: 0.5997 - val_loss: 1.0344 - val_accuracy: 0.6706 - 9s/epoch - 151ms/step
Epoch 23/50
61/61 - 9s - loss: 1.1049 - accuracy: 0.5932 - val_loss: 1.0185 - val_accuracy: 0.6729 - 9s/epoch - 154ms/step
Epoch 24/50
61/61 - 10s - loss: 1.0500 - accuracy: 0.6150 - val_loss: 0.9326 - val_accuracy: 0.7173 - 10s/epoch - 159ms/step
Epoch 25/50
61/61 - 10s - loss: 1.0652 - accuracy: 0.6075 - val_loss: 0.9634 - val_accuracy: 0.7033 - 10s/epoch - 161ms/step
Epoch 26/50
61/61 - 10s - loss: 1.0079 - accuracy: 0.6233 - val_loss: 0.9576 - val_accuracy: 0.6893 - 10s/epoch - 163ms/step
Epoch 27/50
61/61 - 10s - loss: 1.0239 - accuracy: 0.6205 - val_loss: 0.9686 - val_accuracy: 0.6986 - 10s/epoch - 161ms/step
Epoch 28/50
61/61 - 10s - loss: 0.9875 - accuracy: 0.6337 - val_loss: 0.9610 - val_accuracy: 0.6916 - 10s/epoch - 162ms/step
Epoch 29/50
61/61 - 10s - loss: 0.9913 - accuracy: 0.6265 - val_loss: 0.9179 - val_accuracy: 0.7173 - 10s/epoch - 166ms/step
Epoch 30/50
61/61 - 10s - loss: 0.9607 - accuracy: 0.6470 - val_loss: 0.9161 - val_accuracy: 0.7173 - 10s/epoch - 168ms/step
Epoch 31/50
61/61 - 10s - loss: 0.9633 - accuracy: 0.6408 - val_loss: 0.9218 - val_accuracy: 0.7173 - 10s/epoch - 166ms/step
Epoch 32/50
61/61 - 10s - loss: 0.9297 - accuracy: 0.6512 - val_loss: 0.9686 - val_accuracy: 0.6939 - 10s/epoch - 172ms/step
Epoch 33/50
61/61 - 10s - loss: 0.9535 - accuracy: 0.6488 - val_loss: 0.9245 - val_accuracy: 0.7009 - 10s/epoch - 166ms/step
Epoch 34/50
61/61 - 10s - loss: 0.9448 - accuracy: 0.6501 - val_loss: 0.9507 - val_accuracy: 0.7336 - 10s/epoch - 163ms/step
Epoch 35/50
61/61 - 10s - loss: 0.8948 - accuracy: 0.6597 - val_loss: 0.9495 - val_accuracy: 0.7196 - 10s/epoch - 166ms/step
Epoch 36/50
61/61 - 10s - loss: 0.9174 - accuracy: 0.6545 - val_loss: 0.9571 - val_accuracy: 0.7103 - 10s/epoch - 164ms/step
Epoch 37/50
61/61 - 10s - loss: 0.8911 - accuracy: 0.6642 - val_loss: 0.9312 - val_accuracy: 0.7313 - 10s/epoch - 163ms/step
Epoch 38/50
61/61 - 10s - loss: 0.8593 - accuracy: 0.6772 - val_loss: 1.0048 - val_accuracy: 0.7196 - 10s/epoch - 167ms/step
Epoch 39/50
61/61 - 10s - loss: 0.8419 - accuracy: 0.6852 - val_loss: 0.9205 - val_accuracy: 0.7313 - 10s/epoch - 159ms/step
Epoch 40/50
61/61 - 9s - loss: 0.8509 - accuracy: 0.6759 - val_loss: 0.9675 - val_accuracy: 0.7173 - 9s/epoch - 155ms/step
Epoch 41/50
61/61 - 10s - loss: 0.8745 - accuracy: 0.6740 - val_loss: 1.0010 - val_accuracy: 0.7243 - 10s/epoch - 161ms/step
Epoch 42/50
61/61 - 10s - loss: 0.8531 - accuracy: 0.6777 - val_loss: 0.9933 - val_accuracy: 0.7220 - 10s/epoch - 166ms/step
Epoch 43/50
61/61 - 10s - loss: 0.8445 - accuracy: 0.6860 - val_loss: 0.9724 - val_accuracy: 0.7150 - 10s/epoch - 166ms/step
Epoch 44/50
61/61 - 10s - loss: 0.8401 - accuracy: 0.6798 - val_loss: 0.9318 - val_accuracy: 0.7407 - 10s/epoch - 169ms/step
Epoch 45/50
61/61 - 11s - loss: 0.8392 - accuracy: 0.6831 - val_loss: 0.9313 - val_accuracy: 0.7570 - 11s/epoch - 174ms/step
Epoch 46/50
61/61 - 10s - loss: 0.8424 - accuracy: 0.6805 - val_loss: 0.9006 - val_accuracy: 0.7640 - 10s/epoch - 162ms/step
Epoch 47/50
61/61 - 10s - loss: 0.8013 - accuracy: 0.6972 - val_loss: 0.9501 - val_accuracy: 0.7383 - 10s/epoch - 160ms/step
Epoch 48/50
61/61 - 10s - loss: 0.8121 - accuracy: 0.6870 - val_loss: 1.0440 - val_accuracy: 0.7220 - 10s/epoch - 160ms/step
Epoch 49/50
61/61 - 10s - loss: 0.8200 - accuracy: 0.6852 - val_loss: 0.9955 - val_accuracy: 0.7243 - 10s/epoch - 167ms/step
Epoch 50/50
61/61 - 10s - loss: 0.7776 - accuracy: 0.7000 - val_loss: 0.9834 - val_accuracy: 0.7336 - 10s/epoch - 165ms/step

Model Evaluation

plt.plot(base_history.history['accuracy'])
plt.plot(base_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Fom the graph we can see that the training accuracy of the model was reached the 70% range and the validation accuracy was divergent and eventually reached the 60% range.
• It does not seem to improve much past epoch 25.

Loss

plt.plot(base_history.history['loss'])
plt.plot(base_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Loss between Training and validation swaps around epoch 34.
• It does not seem to improve much past epoch 35.

Evaluate the model on test data

base_history_accuracy = base_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 0s 27ms/step - loss: 0.9205 - accuracy: 0.7326

best_model_accuracy = base_history.history['accuracy'][np.argmin(base_history.history['loss'])]
best_model_accuracy

0.7000259757041931

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = base_history_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.700026
Name: Base Model, dtype: float64
accuracy    0.732632
Name: Base Model, dtype: float64

Plotting the Confusion Matrix

# Here we would get the output as probablities for each category
y_pred=base_model.predict(X_test_normalized)

15/15 [==============================] - 0s 28ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• We can observe that some of the plant classes are not predicted correctly.
• It appears none of the Black-grass samples were correctly identified (23, 3).
• We can also observe that Black-grass(26), Common Chickweed(12), Common Wheat(15), Maize(10), Scentless Mayweed(11), Sheperds Purse(10) and Fat Hen(9) have the largest misclassifications.

Complex CNN model containing multiple layers for image processing and dense layer for classification

Here we will add complexity to the model to try and get better results.

CNN Model layers:

• Convolutional input layer, 32 feature maps with a size of 3X3 and a * rectifier activation function
• Max Pool layer with size 2×2 and a stride of 2
• Convolutional layer, 64 feature maps with a size of 3X3 and a rectifier activation function.
• Max Pool layer with size 2×2 and a stride of 2
• Convolutional layer, 128 feature maps with a size of 3X3 and a rectifier activation function.
• Max Pool layer with size 2×2 and a stride of 2
• Convolutional layer, 128 feature maps with a size of 3X3 and a rectifier activation function.
• Max Pool layer with size 2×2 and a stride of 2
• Flatten layer
• Fully connected or Dense layers (with 128 neurons) with Relu Act.
• Dropout layer to reduce overfitting or for regularization
• Fully connected or Dense layers (with 512 neurons) with Relu Act.
• Dropout layer to reduce overfitting or for regularization
• Fully connected or Dense layers (with 256 neurons) with Relu Act.
• Dropout layer to reduce overfitting or for regularization
• Output layer with Softmax to detect multiple categories (12).

# Clearing backend
backend.clear_session()

model_name = "Complex Model"

# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# verify image sizes for input layer
np.shape(X_train_normalized)

(3847, 64, 64, 3)

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
complex_model = Sequential()

# create convolution and max-pooling layers with input size of images
complex_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
complex_model.add(Flatten())

# add a fully connected layer
complex_model.add(Dense(128, activation='relu'))
complex_model.add(Dropout(0.3))

# add a fully connected layer
complex_model.add(Dense(512, activation='relu'))
complex_model.add(Dropout(0.3))

# add a fully connected layer
complex_model.add(Dense(256, activation='relu'))
complex_model.add(Dropout(0.3))

# output layer with 12 neurons and softmax
complex_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
complex_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
complex_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 703564 (2.68 MB)
Trainable params: 703564 (2.68 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

Fitting the model on the train data

complex_history = complex_model.fit(
            X_train_normalized, y_train_encoded,
            epochs=50,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=64,
            verbose=2
)

Epoch 1/50
61/61 - 7s - loss: 2.4347 - accuracy: 0.1318 - val_loss: 2.4040 - val_accuracy: 0.1379 - 7s/epoch - 113ms/step
Epoch 2/50
61/61 - 5s - loss: 2.3157 - accuracy: 0.1994 - val_loss: 1.9667 - val_accuracy: 0.2757 - 5s/epoch - 89ms/step
Epoch 3/50
61/61 - 6s - loss: 1.8033 - accuracy: 0.3384 - val_loss: 1.5833 - val_accuracy: 0.4299 - 6s/epoch - 93ms/step
Epoch 4/50
61/61 - 5s - loss: 1.5718 - accuracy: 0.4359 - val_loss: 1.4064 - val_accuracy: 0.4813 - 5s/epoch - 89ms/step
Epoch 5/50
61/61 - 6s - loss: 1.3404 - accuracy: 0.5269 - val_loss: 1.3290 - val_accuracy: 0.5444 - 6s/epoch - 98ms/step
Epoch 6/50
61/61 - 6s - loss: 1.2357 - accuracy: 0.5719 - val_loss: 1.1874 - val_accuracy: 0.6145 - 6s/epoch - 99ms/step
Epoch 7/50
61/61 - 6s - loss: 1.0727 - accuracy: 0.6192 - val_loss: 1.0276 - val_accuracy: 0.6402 - 6s/epoch - 96ms/step
Epoch 8/50
61/61 - 6s - loss: 1.0283 - accuracy: 0.6371 - val_loss: 1.0235 - val_accuracy: 0.6495 - 6s/epoch - 97ms/step
Epoch 9/50
61/61 - 6s - loss: 0.8854 - accuracy: 0.6875 - val_loss: 0.8745 - val_accuracy: 0.7103 - 6s/epoch - 98ms/step
Epoch 10/50
61/61 - 6s - loss: 0.8156 - accuracy: 0.7070 - val_loss: 0.8217 - val_accuracy: 0.7196 - 6s/epoch - 100ms/step
Epoch 11/50
61/61 - 6s - loss: 0.7527 - accuracy: 0.7325 - val_loss: 0.8224 - val_accuracy: 0.7430 - 6s/epoch - 100ms/step
Epoch 12/50
61/61 - 6s - loss: 0.6732 - accuracy: 0.7637 - val_loss: 0.7062 - val_accuracy: 0.7757 - 6s/epoch - 105ms/step
Epoch 13/50
61/61 - 6s - loss: 0.6550 - accuracy: 0.7702 - val_loss: 0.6765 - val_accuracy: 0.7780 - 6s/epoch - 102ms/step
Epoch 14/50
61/61 - 6s - loss: 0.5796 - accuracy: 0.7954 - val_loss: 0.7914 - val_accuracy: 0.7570 - 6s/epoch - 101ms/step
Epoch 15/50
61/61 - 6s - loss: 0.5378 - accuracy: 0.8144 - val_loss: 0.7139 - val_accuracy: 0.7804 - 6s/epoch - 100ms/step
Epoch 16/50
61/61 - 6s - loss: 0.4865 - accuracy: 0.8253 - val_loss: 0.8664 - val_accuracy: 0.7617 - 6s/epoch - 98ms/step
Epoch 17/50
61/61 - 6s - loss: 0.5135 - accuracy: 0.8217 - val_loss: 0.7266 - val_accuracy: 0.7874 - 6s/epoch - 99ms/step
Epoch 18/50
61/61 - 6s - loss: 0.4516 - accuracy: 0.8407 - val_loss: 0.7053 - val_accuracy: 0.8061 - 6s/epoch - 98ms/step
Epoch 19/50
61/61 - 6s - loss: 0.3818 - accuracy: 0.8646 - val_loss: 0.7122 - val_accuracy: 0.8037 - 6s/epoch - 99ms/step
Epoch 20/50
61/61 - 6s - loss: 0.3534 - accuracy: 0.8734 - val_loss: 0.9190 - val_accuracy: 0.7500 - 6s/epoch - 104ms/step
Epoch 21/50
61/61 - 6s - loss: 0.4036 - accuracy: 0.8568 - val_loss: 0.5883 - val_accuracy: 0.8528 - 6s/epoch - 101ms/step
Epoch 22/50
61/61 - 6s - loss: 0.3096 - accuracy: 0.8906 - val_loss: 0.6213 - val_accuracy: 0.8224 - 6s/epoch - 101ms/step
Epoch 23/50
61/61 - 6s - loss: 0.2988 - accuracy: 0.8924 - val_loss: 0.5923 - val_accuracy: 0.8388 - 6s/epoch - 97ms/step
Epoch 24/50
61/61 - 6s - loss: 0.2919 - accuracy: 0.8984 - val_loss: 0.6838 - val_accuracy: 0.8294 - 6s/epoch - 101ms/step
Epoch 25/50
61/61 - 6s - loss: 0.2891 - accuracy: 0.8916 - val_loss: 0.6880 - val_accuracy: 0.8318 - 6s/epoch - 94ms/step
Epoch 26/50
61/61 - 6s - loss: 0.2444 - accuracy: 0.9124 - val_loss: 0.6306 - val_accuracy: 0.8458 - 6s/epoch - 98ms/step
Epoch 27/50
61/61 - 6s - loss: 0.2725 - accuracy: 0.9038 - val_loss: 0.6195 - val_accuracy: 0.8411 - 6s/epoch - 93ms/step
Epoch 28/50
61/61 - 6s - loss: 0.2339 - accuracy: 0.9153 - val_loss: 0.7309 - val_accuracy: 0.8107 - 6s/epoch - 105ms/step
Epoch 29/50
61/61 - 7s - loss: 0.2325 - accuracy: 0.9121 - val_loss: 0.6667 - val_accuracy: 0.8435 - 7s/epoch - 108ms/step
Epoch 30/50
61/61 - 6s - loss: 0.2069 - accuracy: 0.9218 - val_loss: 0.7933 - val_accuracy: 0.8107 - 6s/epoch - 103ms/step
Epoch 31/50
61/61 - 6s - loss: 0.2072 - accuracy: 0.9259 - val_loss: 0.7165 - val_accuracy: 0.8411 - 6s/epoch - 99ms/step
Epoch 32/50
61/61 - 6s - loss: 0.1889 - accuracy: 0.9288 - val_loss: 0.6839 - val_accuracy: 0.8435 - 6s/epoch - 104ms/step
Epoch 33/50
61/61 - 6s - loss: 0.1845 - accuracy: 0.9311 - val_loss: 0.9491 - val_accuracy: 0.8154 - 6s/epoch - 101ms/step
Epoch 34/50
61/61 - 6s - loss: 0.2155 - accuracy: 0.9257 - val_loss: 0.6805 - val_accuracy: 0.8388 - 6s/epoch - 95ms/step
Epoch 35/50
61/61 - 6s - loss: 0.1635 - accuracy: 0.9392 - val_loss: 0.7970 - val_accuracy: 0.8224 - 6s/epoch - 97ms/step
Epoch 36/50
61/61 - 6s - loss: 0.1609 - accuracy: 0.9446 - val_loss: 0.7326 - val_accuracy: 0.8411 - 6s/epoch - 104ms/step
Epoch 37/50
61/61 - 6s - loss: 0.1809 - accuracy: 0.9376 - val_loss: 0.7756 - val_accuracy: 0.8294 - 6s/epoch - 104ms/step
Epoch 38/50
61/61 - 6s - loss: 0.1577 - accuracy: 0.9459 - val_loss: 0.7072 - val_accuracy: 0.8341 - 6s/epoch - 103ms/step
Epoch 39/50
61/61 - 6s - loss: 0.1201 - accuracy: 0.9594 - val_loss: 0.7259 - val_accuracy: 0.8435 - 6s/epoch - 101ms/step
Epoch 40/50
61/61 - 6s - loss: 0.1136 - accuracy: 0.9584 - val_loss: 0.8301 - val_accuracy: 0.8294 - 6s/epoch - 92ms/step
Epoch 41/50
61/61 - 6s - loss: 0.1585 - accuracy: 0.9465 - val_loss: 0.9825 - val_accuracy: 0.8107 - 6s/epoch - 93ms/step
Epoch 42/50
61/61 - 6s - loss: 0.1818 - accuracy: 0.9363 - val_loss: 0.7074 - val_accuracy: 0.8528 - 6s/epoch - 96ms/step
Epoch 43/50
61/61 - 6s - loss: 0.1364 - accuracy: 0.9535 - val_loss: 0.8272 - val_accuracy: 0.8411 - 6s/epoch - 99ms/step
Epoch 44/50
61/61 - 6s - loss: 0.1742 - accuracy: 0.9441 - val_loss: 0.7004 - val_accuracy: 0.8551 - 6s/epoch - 100ms/step
Epoch 45/50
61/61 - 6s - loss: 0.1075 - accuracy: 0.9639 - val_loss: 0.7788 - val_accuracy: 0.8598 - 6s/epoch - 105ms/step
Epoch 46/50
61/61 - 6s - loss: 0.1207 - accuracy: 0.9576 - val_loss: 0.6841 - val_accuracy: 0.8575 - 6s/epoch - 102ms/step
Epoch 47/50
61/61 - 7s - loss: 0.0969 - accuracy: 0.9665 - val_loss: 0.7042 - val_accuracy: 0.8388 - 7s/epoch - 108ms/step
Epoch 48/50
61/61 - 6s - loss: 0.0831 - accuracy: 0.9711 - val_loss: 0.6746 - val_accuracy: 0.8505 - 6s/epoch - 103ms/step
Epoch 49/50
61/61 - 6s - loss: 0.0955 - accuracy: 0.9670 - val_loss: 0.9756 - val_accuracy: 0.8364 - 6s/epoch - 99ms/step
Epoch 50/50
61/61 - 7s - loss: 0.1566 - accuracy: 0.9504 - val_loss: 0.6661 - val_accuracy: 0.8715 - 7s/epoch - 110ms/step

Model Evaluation

plt.plot(complex_history.history['accuracy'])
plt.plot(complex_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Fom the graph we can see that the training accuracy of the model was improved over the base model and the validation accuracy was also improved.
• The model seems to overfit on the training data.
• It does not seem to improve much past epoch 25. We will use 25 as our base epochs.

Loss

plt.plot(complex_history.history['loss'])
plt.plot(complex_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Loss between Training and validation diverges around epoch 12.
• Validation loss holds fairly constant.
• It does not seem to improve much past epoch 25.

Evaluate the model on test data

complex_history_accuracy = complex_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 38ms/step - loss: 0.7007 - accuracy: 0.8211

best_model_accuracy = complex_history.history['accuracy'][np.argmin(complex_history.history['loss'])]
best_model_accuracy

0.9711463451385498

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = complex_history_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.971146
Name: Complex Model, dtype: float64
accuracy    0.821053
Name: Complex Model, dtype: float64

Observations

• Test Accuracy is 82%
• Training model accuracy with least loss is 97%, showing overfitting.

Plotting the Confusion Matrix

# Here we would get the output as probablities for each category
y_pred=complex_model.predict(X_test_normalized)

15/15 [==============================] - 1s 39ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• We can observe that some of the plant classes are still not predicted correctly.
• It appears that we now have some of the Black-grass samples correctly identified.
• Common Chickweed has dropped from 12 misidentified to 3.
• Common Wheat has dropped from 15 misidentified to 6.
• Fat Hen has dropped from 9 misidentified to 8.
• Maize has dropped from 10 misidentified to 3.
• Sheperds Purse has dropped from 10 misidentified to 5.
• Scentless Mayweed has increased from 11 misidentified to 12.
• Loose Silky-bent remains 5, but it is now misidentified differently.
• Black-grass(22), Common Wheat and Scentless Mayweed have the largest misclassifications.

Model Performance Improvement

Applying Class weights to complex model

from sklearn.utils import class_weight

labelList = np.unique(labels)
class_weights = class_weight.compute_class_weight(class_weight = "balanced",
                                                  classes = np.array(labelList),
                                                  y = y_train.values.reshape(-1)
                                                  )
class_weights = dict(zip(np.array(range(len(labelList))), class_weights))
#print calculated class weights
class_weights

{0: 1.5050860719874803,
 1: 1.0145042194092826,
 2: 1.3818247126436782,
 3: 0.6476430976430977,
 4: 1.7909683426443204,
 5: 0.8348524305555556,
 6: 0.604874213836478,
 7: 1.7909683426443204,
 8: 0.7669457735247209,
 9: 1.714349376114082,
 10: 0.7974709784411277,
 11: 1.0275106837606838}

# Clearing backend
backend.clear_session()

model_name = "Complex Model with Class Weights applied"

# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# verify image sizes for input layer
np.shape(X_train_normalized)

(3847, 64, 64, 3)

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
complex_model_class_weights = Sequential()

# create convolution and max-pooling layers with input size of images
complex_model_class_weights.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model_class_weights.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model_class_weights.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model_class_weights.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
complex_model_class_weights.add(Flatten())

# add a fully connected layer
complex_model_class_weights.add(Dense(128, activation='relu'))
complex_model_class_weights.add(Dropout(0.3))

# add a fully connected layer
complex_model_class_weights.add(Dense(512, activation='relu'))
complex_model_class_weights.add(Dropout(0.3))

# add a fully connected layer
complex_model_class_weights.add(Dense(256, activation='relu'))
complex_model_class_weights.add(Dropout(0.3))

# output layer with 12 neurons and softmax
complex_model_class_weights.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
complex_model_class_weights.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
complex_model_class_weights.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 703564 (2.68 MB)
Trainable params: 703564 (2.68 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

Fitting the model on the train data

complex_class_weights_history = complex_model_class_weights.fit(
            X_train_normalized, y_train_encoded,
            class_weight=class_weights,
            epochs=50,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=64,
            verbose=2
)

Epoch 1/50
61/61 - 8s - loss: 2.4733 - accuracy: 0.0894 - val_loss: 2.4177 - val_accuracy: 0.1215 - 8s/epoch - 132ms/step
Epoch 2/50
61/61 - 6s - loss: 2.1632 - accuracy: 0.1734 - val_loss: 1.9372 - val_accuracy: 0.2383 - 6s/epoch - 100ms/step
Epoch 3/50
61/61 - 6s - loss: 1.8823 - accuracy: 0.2727 - val_loss: 1.7028 - val_accuracy: 0.2664 - 6s/epoch - 101ms/step
Epoch 4/50
61/61 - 6s - loss: 1.7305 - accuracy: 0.3140 - val_loss: 1.7540 - val_accuracy: 0.3271 - 6s/epoch - 102ms/step
Epoch 5/50
61/61 - 6s - loss: 1.7018 - accuracy: 0.3267 - val_loss: 1.5707 - val_accuracy: 0.3949 - 6s/epoch - 99ms/step
Epoch 6/50
61/61 - 6s - loss: 1.5672 - accuracy: 0.3800 - val_loss: 1.5150 - val_accuracy: 0.4346 - 6s/epoch - 98ms/step
Epoch 7/50
61/61 - 6s - loss: 1.5019 - accuracy: 0.4079 - val_loss: 1.3537 - val_accuracy: 0.5070 - 6s/epoch - 97ms/step
Epoch 8/50
61/61 - 6s - loss: 1.4124 - accuracy: 0.4713 - val_loss: 1.2559 - val_accuracy: 0.5678 - 6s/epoch - 97ms/step
Epoch 9/50
61/61 - 6s - loss: 1.2504 - accuracy: 0.5292 - val_loss: 1.1399 - val_accuracy: 0.5607 - 6s/epoch - 94ms/step
Epoch 10/50
61/61 - 6s - loss: 1.1569 - accuracy: 0.5584 - val_loss: 1.0578 - val_accuracy: 0.6565 - 6s/epoch - 92ms/step
Epoch 11/50
61/61 - 6s - loss: 1.0386 - accuracy: 0.6239 - val_loss: 0.9669 - val_accuracy: 0.6752 - 6s/epoch - 91ms/step
Epoch 12/50
61/61 - 6s - loss: 0.9406 - accuracy: 0.6501 - val_loss: 1.1948 - val_accuracy: 0.5888 - 6s/epoch - 93ms/step
Epoch 13/50
61/61 - 6s - loss: 0.8967 - accuracy: 0.6623 - val_loss: 0.7890 - val_accuracy: 0.7617 - 6s/epoch - 99ms/step
Epoch 14/50
61/61 - 6s - loss: 0.7777 - accuracy: 0.7057 - val_loss: 0.9701 - val_accuracy: 0.6799 - 6s/epoch - 101ms/step
Epoch 15/50
61/61 - 6s - loss: 0.7679 - accuracy: 0.7159 - val_loss: 0.8432 - val_accuracy: 0.7336 - 6s/epoch - 98ms/step
Epoch 16/50
61/61 - 6s - loss: 0.6790 - accuracy: 0.7421 - val_loss: 1.0318 - val_accuracy: 0.6822 - 6s/epoch - 106ms/step
Epoch 17/50
61/61 - 6s - loss: 0.6680 - accuracy: 0.7502 - val_loss: 0.8275 - val_accuracy: 0.7453 - 6s/epoch - 100ms/step
Epoch 18/50
61/61 - 6s - loss: 0.6283 - accuracy: 0.7549 - val_loss: 0.7667 - val_accuracy: 0.7757 - 6s/epoch - 97ms/step
Epoch 19/50
61/61 - 6s - loss: 0.5610 - accuracy: 0.7796 - val_loss: 0.7019 - val_accuracy: 0.7921 - 6s/epoch - 98ms/step
Epoch 20/50
61/61 - 6s - loss: 0.5165 - accuracy: 0.8001 - val_loss: 0.7419 - val_accuracy: 0.7780 - 6s/epoch - 95ms/step
Epoch 21/50
61/61 - 6s - loss: 0.4671 - accuracy: 0.8209 - val_loss: 0.8504 - val_accuracy: 0.7313 - 6s/epoch - 93ms/step
Epoch 22/50
61/61 - 6s - loss: 0.4751 - accuracy: 0.8175 - val_loss: 0.9202 - val_accuracy: 0.7383 - 6s/epoch - 93ms/step
Epoch 23/50
61/61 - 6s - loss: 0.4279 - accuracy: 0.8339 - val_loss: 0.8281 - val_accuracy: 0.7383 - 6s/epoch - 91ms/step
Epoch 24/50
61/61 - 6s - loss: 0.4244 - accuracy: 0.8225 - val_loss: 0.7795 - val_accuracy: 0.7664 - 6s/epoch - 95ms/step
Epoch 25/50
61/61 - 6s - loss: 0.4738 - accuracy: 0.8173 - val_loss: 0.7374 - val_accuracy: 0.8014 - 6s/epoch - 94ms/step
Epoch 26/50
61/61 - 6s - loss: 0.4435 - accuracy: 0.8355 - val_loss: 0.8845 - val_accuracy: 0.7523 - 6s/epoch - 98ms/step
Epoch 27/50
61/61 - 6s - loss: 0.3591 - accuracy: 0.8544 - val_loss: 0.7173 - val_accuracy: 0.7921 - 6s/epoch - 98ms/step
Epoch 28/50
61/61 - 6s - loss: 0.3211 - accuracy: 0.8713 - val_loss: 0.7383 - val_accuracy: 0.7967 - 6s/epoch - 100ms/step
Epoch 29/50
61/61 - 6s - loss: 0.2918 - accuracy: 0.8846 - val_loss: 0.7289 - val_accuracy: 0.8131 - 6s/epoch - 97ms/step
Epoch 30/50
61/61 - 6s - loss: 0.3121 - accuracy: 0.8783 - val_loss: 0.7548 - val_accuracy: 0.7944 - 6s/epoch - 101ms/step
Epoch 31/50
61/61 - 6s - loss: 0.3089 - accuracy: 0.8783 - val_loss: 0.6678 - val_accuracy: 0.8224 - 6s/epoch - 95ms/step
Epoch 32/50
61/61 - 6s - loss: 0.2557 - accuracy: 0.8976 - val_loss: 0.9462 - val_accuracy: 0.7640 - 6s/epoch - 93ms/step
Epoch 33/50
61/61 - 6s - loss: 0.2774 - accuracy: 0.8900 - val_loss: 0.8480 - val_accuracy: 0.7827 - 6s/epoch - 101ms/step
Epoch 34/50
61/61 - 6s - loss: 0.2505 - accuracy: 0.8968 - val_loss: 0.7831 - val_accuracy: 0.8084 - 6s/epoch - 102ms/step
Epoch 35/50
61/61 - 6s - loss: 0.2694 - accuracy: 0.8950 - val_loss: 0.7323 - val_accuracy: 0.8154 - 6s/epoch - 99ms/step
Epoch 36/50
61/61 - 6s - loss: 0.2176 - accuracy: 0.9101 - val_loss: 0.7685 - val_accuracy: 0.8107 - 6s/epoch - 98ms/step
Epoch 37/50
61/61 - 6s - loss: 0.2277 - accuracy: 0.9129 - val_loss: 0.7944 - val_accuracy: 0.7991 - 6s/epoch - 100ms/step
Epoch 38/50
61/61 - 6s - loss: 0.2324 - accuracy: 0.8929 - val_loss: 0.7791 - val_accuracy: 0.8201 - 6s/epoch - 100ms/step
Epoch 39/50
61/61 - 6s - loss: 0.2752 - accuracy: 0.8890 - val_loss: 0.8073 - val_accuracy: 0.8061 - 6s/epoch - 95ms/step
Epoch 40/50
61/61 - 6s - loss: 0.2179 - accuracy: 0.9095 - val_loss: 0.7444 - val_accuracy: 0.8318 - 6s/epoch - 96ms/step
Epoch 41/50
61/61 - 6s - loss: 0.2212 - accuracy: 0.9132 - val_loss: 0.8191 - val_accuracy: 0.8131 - 6s/epoch - 98ms/step
Epoch 42/50
61/61 - 6s - loss: 0.1954 - accuracy: 0.9186 - val_loss: 0.7774 - val_accuracy: 0.8294 - 6s/epoch - 93ms/step
Epoch 43/50
61/61 - 6s - loss: 0.1977 - accuracy: 0.9283 - val_loss: 0.7761 - val_accuracy: 0.7991 - 6s/epoch - 92ms/step
Epoch 44/50
61/61 - 6s - loss: 0.1874 - accuracy: 0.9238 - val_loss: 0.8406 - val_accuracy: 0.8107 - 6s/epoch - 96ms/step
Epoch 45/50
61/61 - 6s - loss: 0.1646 - accuracy: 0.9329 - val_loss: 0.7810 - val_accuracy: 0.8294 - 6s/epoch - 94ms/step
Epoch 46/50
61/61 - 6s - loss: 0.1658 - accuracy: 0.9363 - val_loss: 0.8834 - val_accuracy: 0.7944 - 6s/epoch - 92ms/step
Epoch 47/50
61/61 - 6s - loss: 0.1659 - accuracy: 0.9288 - val_loss: 0.9497 - val_accuracy: 0.8224 - 6s/epoch - 96ms/step
Epoch 48/50
61/61 - 6s - loss: 0.1735 - accuracy: 0.9303 - val_loss: 0.7504 - val_accuracy: 0.8178 - 6s/epoch - 96ms/step
Epoch 49/50
61/61 - 6s - loss: 0.1488 - accuracy: 0.9392 - val_loss: 0.8148 - val_accuracy: 0.8178 - 6s/epoch - 93ms/step
Epoch 50/50
61/61 - 6s - loss: 0.1673 - accuracy: 0.9392 - val_loss: 0.8270 - val_accuracy: 0.8154 - 6s/epoch - 99ms/step

Model Evaluation

plt.plot(complex_class_weights_history.history['accuracy'])
plt.plot(complex_class_weights_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Fom the graph we can see that the training accuracy of the model was improved over the base model and the validation accuracy was also improved.
• The model seems to overfit on the training data.
• It does not seem to improve much past epoch 25. We will use 25 as our base epochs.

Loss

plt.plot(complex_class_weights_history.history['loss'])
plt.plot(complex_class_weights_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Loss between Training and validation diverges around epoch 12.
• Validation loss holds fairly constant.
• It does not seem to improve much past epoch 25.

Evaluate the model on test data

complex_class_weights_history_accuracy = complex_model_class_weights.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 39ms/step - loss: 0.8297 - accuracy: 0.7958

best_model_accuracy = complex_class_weights_history.history['accuracy'][np.argmin(complex_class_weights_history.history['loss'])]
best_model_accuracy

0.9391734004020691

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = complex_class_weights_history_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.939173
Name: Complex Model with Class Weights applied, dtype: float64
accuracy    0.795789
Name: Complex Model with Class Weights applied, dtype: float64

Observations

• Test Accuracy is 80%
• Training model accuracy with least loss is 94%.
• Adjusting the class weights did not improve identification.

Plotting the Confusion Matrix

# Here we would get the output as probablities for each category
y_pred=complex_model.predict(X_test_normalized)

15/15 [==============================] - 1s 40ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• The misidentified plants did not improve much, if at all. We will try another strategy.

Model using adjustment of the Adam optimizer:

In this model we will not use Class weights. We will adjust the adam optimizer using learning rate, decay rate and avoiding division by zero.

# Clearing backend
backend.clear_session()

model_name = "Complex Model with Adam optimizer adjusted"

# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# verify image sizes for input layer
np.shape(X_train_normalized)

(3847, 64, 64, 3)

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
complex_model_adam_opt = Sequential()

# create convolution and max-pooling layers with input size of images
complex_model_adam_opt.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model_adam_opt.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model_adam_opt.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
complex_model_adam_opt.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
complex_model_adam_opt.add(Flatten())

# add a fully connected layer
complex_model_adam_opt.add(Dense(128, activation='relu'))
complex_model_adam_opt.add(Dropout(0.3))

# add a fully connected layer
complex_model_adam_opt.add(Dense(512, activation='relu'))
complex_model_adam_opt.add(Dropout(0.3))

# add a fully connected layer
complex_model_adam_opt.add(Dense(256, activation='relu'))
complex_model_adam_opt.add(Dropout(0.3))

# output layer with 12 neurons and softmax
complex_model_adam_opt.add(Dense(12, activation='softmax'))

# Adam Optimizer
adam_opt = optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)

# Compile the model
complex_model_adam_opt.compile(optimizer=adam_opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
complex_model_adam_opt.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 703564 (2.68 MB)
Trainable params: 703564 (2.68 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

Fitting the model on the train data

complex_adam_opt_history = complex_model_adam_opt.fit(
            X_train_normalized, y_train_encoded,
            epochs=50,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=64,
            verbose=2
)

Epoch 1/50
61/61 - 7s - loss: 2.4335 - accuracy: 0.1300 - val_loss: 2.4119 - val_accuracy: 0.1379 - 7s/epoch - 123ms/step
Epoch 2/50
61/61 - 6s - loss: 2.4147 - accuracy: 0.1300 - val_loss: 2.4252 - val_accuracy: 0.1285 - 6s/epoch - 105ms/step
Epoch 3/50
61/61 - 6s - loss: 2.4214 - accuracy: 0.1398 - val_loss: 2.4081 - val_accuracy: 0.1379 - 6s/epoch - 100ms/step
Epoch 4/50
61/61 - 6s - loss: 2.3789 - accuracy: 0.1521 - val_loss: 2.3051 - val_accuracy: 0.1612 - 6s/epoch - 104ms/step
Epoch 5/50
61/61 - 6s - loss: 2.1494 - accuracy: 0.2313 - val_loss: 1.9198 - val_accuracy: 0.3551 - 6s/epoch - 106ms/step
Epoch 6/50
61/61 - 6s - loss: 1.7694 - accuracy: 0.3543 - val_loss: 1.5920 - val_accuracy: 0.4229 - 6s/epoch - 103ms/step
Epoch 7/50
61/61 - 6s - loss: 1.5474 - accuracy: 0.4474 - val_loss: 1.4632 - val_accuracy: 0.5117 - 6s/epoch - 101ms/step
Epoch 8/50
61/61 - 7s - loss: 1.3990 - accuracy: 0.5069 - val_loss: 1.1936 - val_accuracy: 0.5888 - 7s/epoch - 107ms/step
Epoch 9/50
61/61 - 7s - loss: 1.2051 - accuracy: 0.5688 - val_loss: 1.4414 - val_accuracy: 0.5070 - 7s/epoch - 107ms/step
Epoch 10/50
61/61 - 6s - loss: 1.1873 - accuracy: 0.5763 - val_loss: 1.1966 - val_accuracy: 0.5794 - 6s/epoch - 103ms/step
Epoch 11/50
61/61 - 6s - loss: 1.0036 - accuracy: 0.6428 - val_loss: 1.0713 - val_accuracy: 0.6495 - 6s/epoch - 104ms/step
Epoch 12/50
61/61 - 6s - loss: 0.9437 - accuracy: 0.6688 - val_loss: 0.9207 - val_accuracy: 0.6729 - 6s/epoch - 98ms/step
Epoch 13/50
61/61 - 7s - loss: 0.8662 - accuracy: 0.6855 - val_loss: 0.8806 - val_accuracy: 0.7126 - 7s/epoch - 110ms/step
Epoch 14/50
61/61 - 7s - loss: 0.8069 - accuracy: 0.7120 - val_loss: 0.8839 - val_accuracy: 0.7173 - 7s/epoch - 110ms/step
Epoch 15/50
61/61 - 7s - loss: 0.7473 - accuracy: 0.7356 - val_loss: 0.8143 - val_accuracy: 0.7430 - 7s/epoch - 107ms/step
Epoch 16/50
61/61 - 6s - loss: 0.7139 - accuracy: 0.7466 - val_loss: 0.8805 - val_accuracy: 0.7290 - 6s/epoch - 106ms/step
Epoch 17/50
61/61 - 7s - loss: 0.6794 - accuracy: 0.7533 - val_loss: 0.7486 - val_accuracy: 0.7453 - 7s/epoch - 107ms/step
Epoch 18/50
61/61 - 7s - loss: 0.6679 - accuracy: 0.7632 - val_loss: 0.7543 - val_accuracy: 0.7523 - 7s/epoch - 107ms/step
Epoch 19/50
61/61 - 6s - loss: 0.5526 - accuracy: 0.7957 - val_loss: 0.7203 - val_accuracy: 0.7757 - 6s/epoch - 104ms/step
Epoch 20/50
61/61 - 6s - loss: 0.5371 - accuracy: 0.8009 - val_loss: 0.9041 - val_accuracy: 0.7360 - 6s/epoch - 104ms/step
Epoch 21/50
61/61 - 6s - loss: 0.5016 - accuracy: 0.8227 - val_loss: 0.6814 - val_accuracy: 0.8061 - 6s/epoch - 104ms/step
Epoch 22/50
61/61 - 6s - loss: 0.4542 - accuracy: 0.8329 - val_loss: 0.6464 - val_accuracy: 0.8061 - 6s/epoch - 99ms/step
Epoch 23/50
61/61 - 6s - loss: 0.4387 - accuracy: 0.8422 - val_loss: 0.6651 - val_accuracy: 0.7921 - 6s/epoch - 102ms/step
Epoch 24/50
61/61 - 6s - loss: 0.4250 - accuracy: 0.8472 - val_loss: 0.7764 - val_accuracy: 0.7710 - 6s/epoch - 103ms/step
Epoch 25/50
61/61 - 7s - loss: 0.3783 - accuracy: 0.8583 - val_loss: 0.6588 - val_accuracy: 0.8154 - 7s/epoch - 109ms/step
Epoch 26/50
61/61 - 6s - loss: 0.3841 - accuracy: 0.8555 - val_loss: 0.6227 - val_accuracy: 0.8201 - 6s/epoch - 103ms/step
Epoch 27/50
61/61 - 6s - loss: 0.3339 - accuracy: 0.8700 - val_loss: 0.6962 - val_accuracy: 0.7944 - 6s/epoch - 100ms/step
Epoch 28/50
61/61 - 6s - loss: 0.3126 - accuracy: 0.8830 - val_loss: 0.6416 - val_accuracy: 0.8318 - 6s/epoch - 102ms/step
Epoch 29/50
61/61 - 6s - loss: 0.3219 - accuracy: 0.8822 - val_loss: 0.7842 - val_accuracy: 0.7897 - 6s/epoch - 97ms/step
Epoch 30/50
61/61 - 6s - loss: 0.3281 - accuracy: 0.8799 - val_loss: 0.6454 - val_accuracy: 0.8248 - 6s/epoch - 101ms/step
Epoch 31/50
61/61 - 6s - loss: 0.3063 - accuracy: 0.8843 - val_loss: 0.7224 - val_accuracy: 0.7921 - 6s/epoch - 97ms/step
Epoch 32/50
61/61 - 7s - loss: 0.2605 - accuracy: 0.9015 - val_loss: 0.6440 - val_accuracy: 0.8388 - 7s/epoch - 109ms/step
Epoch 33/50
61/61 - 7s - loss: 0.2490 - accuracy: 0.9015 - val_loss: 0.7124 - val_accuracy: 0.8201 - 7s/epoch - 111ms/step
Epoch 34/50
61/61 - 6s - loss: 0.2567 - accuracy: 0.9012 - val_loss: 0.6405 - val_accuracy: 0.8131 - 6s/epoch - 105ms/step
Epoch 35/50
61/61 - 6s - loss: 0.2626 - accuracy: 0.8997 - val_loss: 0.8338 - val_accuracy: 0.7921 - 6s/epoch - 106ms/step
Epoch 36/50
61/61 - 6s - loss: 0.2509 - accuracy: 0.9043 - val_loss: 0.6245 - val_accuracy: 0.8458 - 6s/epoch - 105ms/step
Epoch 37/50
61/61 - 6s - loss: 0.2209 - accuracy: 0.9186 - val_loss: 0.6883 - val_accuracy: 0.8201 - 6s/epoch - 98ms/step
Epoch 38/50
61/61 - 6s - loss: 0.2431 - accuracy: 0.9093 - val_loss: 0.6749 - val_accuracy: 0.8131 - 6s/epoch - 101ms/step
Epoch 39/50
61/61 - 6s - loss: 0.2325 - accuracy: 0.9124 - val_loss: 0.7944 - val_accuracy: 0.7921 - 6s/epoch - 99ms/step
Epoch 40/50
61/61 - 6s - loss: 0.2552 - accuracy: 0.9038 - val_loss: 0.6227 - val_accuracy: 0.8271 - 6s/epoch - 97ms/step
Epoch 41/50
61/61 - 6s - loss: 0.1817 - accuracy: 0.9332 - val_loss: 0.6876 - val_accuracy: 0.8178 - 6s/epoch - 98ms/step
Epoch 42/50
61/61 - 6s - loss: 0.2605 - accuracy: 0.9028 - val_loss: 0.6727 - val_accuracy: 0.8271 - 6s/epoch - 104ms/step
Epoch 43/50
61/61 - 6s - loss: 0.1990 - accuracy: 0.9254 - val_loss: 0.6308 - val_accuracy: 0.8364 - 6s/epoch - 105ms/step
Epoch 44/50
61/61 - 6s - loss: 0.1914 - accuracy: 0.9322 - val_loss: 0.5838 - val_accuracy: 0.8645 - 6s/epoch - 97ms/step
Epoch 45/50
61/61 - 6s - loss: 0.1593 - accuracy: 0.9410 - val_loss: 0.6850 - val_accuracy: 0.8318 - 6s/epoch - 100ms/step
Epoch 46/50
61/61 - 6s - loss: 0.1828 - accuracy: 0.9327 - val_loss: 0.6906 - val_accuracy: 0.8364 - 6s/epoch - 105ms/step
Epoch 47/50
61/61 - 6s - loss: 0.1739 - accuracy: 0.9355 - val_loss: 0.8667 - val_accuracy: 0.7921 - 6s/epoch - 100ms/step
Epoch 48/50
61/61 - 6s - loss: 0.1701 - accuracy: 0.9374 - val_loss: 0.5989 - val_accuracy: 0.8458 - 6s/epoch - 104ms/step
Epoch 49/50
61/61 - 7s - loss: 0.1656 - accuracy: 0.9410 - val_loss: 0.7510 - val_accuracy: 0.8131 - 7s/epoch - 110ms/step
Epoch 50/50
61/61 - 6s - loss: 0.1375 - accuracy: 0.9491 - val_loss: 0.6773 - val_accuracy: 0.8598 - 6s/epoch - 104ms/step

Model Evaluation

plt.plot(complex_adam_opt_history.history['accuracy'])
plt.plot(complex_adam_opt_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• This model behaves much like the non adjusted adam optimizer model.
• It does not seem to improve much past epoch 25.

Loss

plt.plot(complex_adam_opt_history.history['loss'])
plt.plot(complex_adam_opt_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Again similar performance as the non adjusted adam model.
• It does not seem to improve much past epoch 35.

Evaluate the model on test data

complex_history_adam_opt_accuracy = complex_model_adam_opt.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 41ms/step - loss: 0.6181 - accuracy: 0.8400

best_model_accuracy = complex_adam_opt_history.history['accuracy'][np.argmin(complex_adam_opt_history.history['loss'])]
best_model_accuracy

0.9490512013435364

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = complex_history_adam_opt_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.949051
Name: Complex Model with Adam optimizer adjusted, dtype: float64
accuracy    0.84
Name: Complex Model with Adam optimizer adjusted, dtype: float64

Observations

• Test Accuracy is 84%
• Training model accuracy with least loss is 95% so a bit of overfitting there.

Plotting the Confusion Matrix

# Here we would get the output as probablities for each category
y_pred=complex_model_adam_opt.predict(X_test_normalized)

15/15 [==============================] - 1s 38ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• Common Chickweed, Sheperds Purse, and Common Wheat did not improve from base complex model.
• Fat Hen has improved a bit from 8 misidentified to 6.
• Loose Silky-bent and Maize have been 100% identified correctly.
• Scentless Mayweed returned to 11 misidentified from 12.
• Black-grass(14) remains the largest misclassification.

Model adjusted by reducing the Learning Rate:

In this model we will use the ReduceLRonPlateau to adjust the model. We will also add Batch Normalization.

ReduceLRonPlateau() is a function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.

# Code to monitor val_accuracy
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
                                            patience=2, #was 3
                                            verbose=1,
                                            factor=0.5,
                                            min_lr=0.00001)

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

model_name = "Complex Model with learning rate reduction and batch normalization"

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
lron_base_model = Sequential()

# create convolution and max-pooling layers with input size of images
lron_base_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_base_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_base_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_base_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
lron_base_model.add(Flatten())

# add a fully connected layer
lron_base_model.add(Dense(128, activation='relu'))
lron_base_model.add(Dropout(0.3))

# add a fully connected layer
lron_base_model.add(Dense(512, activation='relu'))
lron_base_model.add(Dropout(0.3))

# add a fully connected layer
lron_base_model.add(Dense(256, activation='relu'))
lron_base_model.add(Dropout(0.3))

# output layer with 12 neurons and softmax
lron_base_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
lron_base_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
lron_base_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 batch_normalization (Batch  (None, 64, 64, 32)        128       
 Normalization)                                                  
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 batch_normalization_1 (Bat  (None, 32, 32, 64)        256       
 chNormalization)                                                
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 batch_normalization_2 (Bat  (None, 16, 16, 128)       512       
 chNormalization)                                                
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 batch_normalization_3 (Bat  (None, 8, 8, 128)         512       
 chNormalization)                                                
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________

Fitting the model on the train data

lron_base_history = lron_base_model.fit(
            X_train_normalized, y_train_encoded,
            epochs=epochs,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=batch_size,
            verbose=2,callbacks=[learning_rate_reduction])

Epoch 1/25
121/121 - 19s - loss: 2.0012 - accuracy: 0.3312 - val_loss: 7.4617 - val_accuracy: 0.1379 - lr: 0.0010 - 19s/epoch - 156ms/step
Epoch 2/25
121/121 - 17s - loss: 1.3346 - accuracy: 0.5545 - val_loss: 13.4252 - val_accuracy: 0.1379 - lr: 0.0010 - 17s/epoch - 139ms/step
Epoch 3/25
121/121 - 16s - loss: 1.0517 - accuracy: 0.6460 - val_loss: 4.0732 - val_accuracy: 0.1869 - lr: 0.0010 - 16s/epoch - 136ms/step
Epoch 4/25
121/121 - 17s - loss: 0.8448 - accuracy: 0.7174 - val_loss: 13.2370 - val_accuracy: 0.1379 - lr: 0.0010 - 17s/epoch - 138ms/step
Epoch 5/25
121/121 - 17s - loss: 0.7431 - accuracy: 0.7515 - val_loss: 3.0628 - val_accuracy: 0.3925 - lr: 0.0010 - 17s/epoch - 139ms/step
Epoch 6/25
121/121 - 16s - loss: 0.6106 - accuracy: 0.7915 - val_loss: 1.8700 - val_accuracy: 0.4883 - lr: 0.0010 - 16s/epoch - 133ms/step
Epoch 7/25
121/121 - 15s - loss: 0.5493 - accuracy: 0.8121 - val_loss: 0.8549 - val_accuracy: 0.7687 - lr: 0.0010 - 15s/epoch - 128ms/step
Epoch 8/25
121/121 - 16s - loss: 0.4791 - accuracy: 0.8352 - val_loss: 1.2403 - val_accuracy: 0.5981 - lr: 0.0010 - 16s/epoch - 134ms/step
Epoch 9/25
121/121 - 18s - loss: 0.4797 - accuracy: 0.8310 - val_loss: 0.6532 - val_accuracy: 0.8107 - lr: 0.0010 - 18s/epoch - 145ms/step
Epoch 10/25
121/121 - 18s - loss: 0.3922 - accuracy: 0.8638 - val_loss: 1.3370 - val_accuracy: 0.5888 - lr: 0.0010 - 18s/epoch - 149ms/step
Epoch 11/25

Epoch 11: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
121/121 - 18s - loss: 0.3685 - accuracy: 0.8695 - val_loss: 0.7262 - val_accuracy: 0.7850 - lr: 0.0010 - 18s/epoch - 152ms/step
Epoch 12/25
121/121 - 19s - loss: 0.2520 - accuracy: 0.9043 - val_loss: 0.5732 - val_accuracy: 0.8435 - lr: 5.0000e-04 - 19s/epoch - 156ms/step
Epoch 13/25
121/121 - 18s - loss: 0.1979 - accuracy: 0.9264 - val_loss: 0.7829 - val_accuracy: 0.8201 - lr: 5.0000e-04 - 18s/epoch - 150ms/step
Epoch 14/25
121/121 - 18s - loss: 0.1824 - accuracy: 0.9324 - val_loss: 0.5219 - val_accuracy: 0.8621 - lr: 5.0000e-04 - 18s/epoch - 145ms/step
Epoch 15/25
121/121 - 18s - loss: 0.1678 - accuracy: 0.9389 - val_loss: 1.3576 - val_accuracy: 0.7033 - lr: 5.0000e-04 - 18s/epoch - 149ms/step
Epoch 16/25

Epoch 16: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
121/121 - 17s - loss: 0.1665 - accuracy: 0.9389 - val_loss: 1.1790 - val_accuracy: 0.7266 - lr: 5.0000e-04 - 17s/epoch - 143ms/step
Epoch 17/25
121/121 - 18s - loss: 0.1293 - accuracy: 0.9530 - val_loss: 0.4625 - val_accuracy: 0.8949 - lr: 2.5000e-04 - 18s/epoch - 146ms/step
Epoch 18/25
121/121 - 17s - loss: 0.1205 - accuracy: 0.9605 - val_loss: 0.6072 - val_accuracy: 0.8668 - lr: 2.5000e-04 - 17s/epoch - 143ms/step
Epoch 19/25

Epoch 19: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
121/121 - 18s - loss: 0.0898 - accuracy: 0.9662 - val_loss: 0.5988 - val_accuracy: 0.8598 - lr: 2.5000e-04 - 18s/epoch - 148ms/step
Epoch 20/25
121/121 - 17s - loss: 0.0773 - accuracy: 0.9748 - val_loss: 0.4249 - val_accuracy: 0.9182 - lr: 1.2500e-04 - 17s/epoch - 138ms/step
Epoch 21/25
121/121 - 16s - loss: 0.0630 - accuracy: 0.9797 - val_loss: 0.4789 - val_accuracy: 0.8925 - lr: 1.2500e-04 - 16s/epoch - 131ms/step
Epoch 22/25

Epoch 22: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
121/121 - 16s - loss: 0.0716 - accuracy: 0.9743 - val_loss: 0.5871 - val_accuracy: 0.8949 - lr: 1.2500e-04 - 16s/epoch - 131ms/step
Epoch 23/25
121/121 - 17s - loss: 0.0584 - accuracy: 0.9800 - val_loss: 0.5589 - val_accuracy: 0.9019 - lr: 6.2500e-05 - 17s/epoch - 143ms/step
Epoch 24/25

Epoch 24: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
121/121 - 17s - loss: 0.0498 - accuracy: 0.9831 - val_loss: 0.4390 - val_accuracy: 0.9182 - lr: 6.2500e-05 - 17s/epoch - 140ms/step
Epoch 25/25
121/121 - 18s - loss: 0.0446 - accuracy: 0.9857 - val_loss: 0.4321 - val_accuracy: 0.9206 - lr: 3.1250e-05 - 18s/epoch - 149ms/step

Model Evaluation

plt.plot(lron_base_history.history['accuracy'])
plt.plot(lron_base_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Fom the graph we can see that the training accuracy of the model was reaching overfitting.
• The validation accuracy was oscillating and divergent.

plt.plot(lron_base_history.history['loss'])
plt.plot(lron_base_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations

• Loss between Training and validation remains pretty stable after epoch 13.

Evaluate the model on test data

lron_base_model_test_accuracy = lron_base_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 62ms/step - loss: 0.4490 - accuracy: 0.9116

best_model_accuracy = lron_base_history.history['accuracy'][np.argmin(lron_base_history.history['loss'])]
best_model_accuracy

0.98570317029953

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_base_model_test_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.985703
Name: Complex Model with learning rate reduction and batch normalization, dtype: float64
accuracy    0.911579
Name: Complex Model with learning rate reduction and batch normalization, dtype: float64

Observations

• Test Accuracy is 91%
• Training model accuracy with least loss is 99%.

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=lron_base_model.predict(X_test_normalized)

15/15 [==============================] - 1s 61ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• Black-grass has improved from 14 to 12 misindentified
• Common Chickweed, Common Wheat, Fat Hen, Scentless Mayweed have all improved.
• Maize has been 100% identified correctly.
• Sheperds Purse has increased in misidentification.
• Loose Silky-bent is still highly misclassified.

Model with basic Data Augmentation

In this model we will add basic data augmentation techniques and keep the batch normalization and learning rate reduction.

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
                              rotation_range=20,
                              fill_mode='nearest'
                              )

model_name = "Complex Model with batch normalization, learning rate reduction and basic data augmentation"

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
lron_da_model = Sequential()

# create convolution and max-pooling layers with input size of images
lron_da_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
lron_da_model.add(Flatten())

# add a fully connected layer
lron_da_model.add(Dense(128, activation='relu'))
lron_da_model.add(Dropout(0.3))

# add a fully connected layer
lron_da_model.add(Dense(512, activation='relu'))
lron_da_model.add(Dropout(0.3))

# add a fully connected layer
lron_da_model.add(Dense(256, activation='relu'))
lron_da_model.add(Dropout(0.3))

# output layer with 12 neurons and softmax
lron_da_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
lron_da_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
lron_da_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 batch_normalization (Batch  (None, 64, 64, 32)        128       
 Normalization)                                                  
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 batch_normalization_1 (Bat  (None, 32, 32, 64)        256       
 chNormalization)                                                
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 batch_normalization_2 (Bat  (None, 16, 16, 128)       512       
 chNormalization)                                                
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 batch_normalization_3 (Bat  (None, 8, 8, 128)         512       
 chNormalization)                                                
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________

Fitting the model on the train data

lron_da_history = lron_da_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=32,
                                       shuffle=False),
                                       epochs=epochs,
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,
                                       callbacks=[learning_rate_reduction])

Epoch 1/25
121/121 [==============================] - 19s 146ms/step - loss: 2.0073 - accuracy: 0.3174 - val_loss: 8.4350 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 2/25
121/121 [==============================] - 19s 154ms/step - loss: 1.3775 - accuracy: 0.5217 - val_loss: 15.4300 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 3/25
121/121 [==============================] - ETA: 0s - loss: 1.1028 - accuracy: 0.6291
Epoch 3: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
121/121 [==============================] - 19s 154ms/step - loss: 1.1028 - accuracy: 0.6291 - val_loss: 15.1994 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 4/25
121/121 [==============================] - 18s 152ms/step - loss: 0.8084 - accuracy: 0.7255 - val_loss: 10.1189 - val_accuracy: 0.1379 - lr: 5.0000e-04
Epoch 5/25
121/121 [==============================] - 19s 153ms/step - loss: 0.6915 - accuracy: 0.7635 - val_loss: 2.3219 - val_accuracy: 0.4463 - lr: 5.0000e-04
Epoch 6/25
121/121 [==============================] - 18s 151ms/step - loss: 0.5962 - accuracy: 0.7926 - val_loss: 1.1982 - val_accuracy: 0.6636 - lr: 5.0000e-04
Epoch 7/25
121/121 [==============================] - 19s 157ms/step - loss: 0.5363 - accuracy: 0.8097 - val_loss: 0.8725 - val_accuracy: 0.7500 - lr: 5.0000e-04
Epoch 8/25
121/121 [==============================] - 17s 138ms/step - loss: 0.5366 - accuracy: 0.8134 - val_loss: 1.0062 - val_accuracy: 0.6939 - lr: 5.0000e-04
Epoch 9/25
121/121 [==============================] - 18s 145ms/step - loss: 0.4544 - accuracy: 0.8378 - val_loss: 0.7697 - val_accuracy: 0.7687 - lr: 5.0000e-04
Epoch 10/25
121/121 [==============================] - 18s 152ms/step - loss: 0.3830 - accuracy: 0.8550 - val_loss: 1.4741 - val_accuracy: 0.5678 - lr: 5.0000e-04
Epoch 11/25
121/121 [==============================] - 20s 165ms/step - loss: 0.3838 - accuracy: 0.8653 - val_loss: 0.5951 - val_accuracy: 0.8528 - lr: 5.0000e-04
Epoch 12/25
121/121 [==============================] - 20s 169ms/step - loss: 0.3689 - accuracy: 0.8718 - val_loss: 0.6033 - val_accuracy: 0.8248 - lr: 5.0000e-04
Epoch 13/25
121/121 [==============================] - 19s 158ms/step - loss: 0.3250 - accuracy: 0.8802 - val_loss: 0.5281 - val_accuracy: 0.8715 - lr: 5.0000e-04
Epoch 14/25
121/121 [==============================] - 19s 158ms/step - loss: 0.2926 - accuracy: 0.8913 - val_loss: 0.6886 - val_accuracy: 0.8084 - lr: 5.0000e-04
Epoch 15/25
121/121 [==============================] - ETA: 0s - loss: 0.3045 - accuracy: 0.8885
Epoch 15: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
121/121 [==============================] - 19s 155ms/step - loss: 0.3045 - accuracy: 0.8885 - val_loss: 0.6094 - val_accuracy: 0.8201 - lr: 5.0000e-04
Epoch 16/25
121/121 [==============================] - 18s 152ms/step - loss: 0.2171 - accuracy: 0.9189 - val_loss: 0.5359 - val_accuracy: 0.8505 - lr: 2.5000e-04
Epoch 17/25
121/121 [==============================] - ETA: 0s - loss: 0.1898 - accuracy: 0.9233
Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
121/121 [==============================] - 18s 152ms/step - loss: 0.1898 - accuracy: 0.9233 - val_loss: 0.7065 - val_accuracy: 0.8178 - lr: 2.5000e-04
Epoch 18/25
121/121 [==============================] - 19s 154ms/step - loss: 0.1758 - accuracy: 0.9361 - val_loss: 0.4001 - val_accuracy: 0.9042 - lr: 1.2500e-04
Epoch 19/25
121/121 [==============================] - 19s 157ms/step - loss: 0.1443 - accuracy: 0.9444 - val_loss: 0.4023 - val_accuracy: 0.9042 - lr: 1.2500e-04
Epoch 20/25
121/121 [==============================] - 19s 158ms/step - loss: 0.1386 - accuracy: 0.9488 - val_loss: 0.4146 - val_accuracy: 0.9112 - lr: 1.2500e-04
Epoch 21/25
121/121 [==============================] - 19s 161ms/step - loss: 0.1340 - accuracy: 0.9488 - val_loss: 0.4319 - val_accuracy: 0.9042 - lr: 1.2500e-04
Epoch 22/25
121/121 [==============================] - ETA: 0s - loss: 0.1251 - accuracy: 0.9509
Epoch 22: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
121/121 [==============================] - 19s 156ms/step - loss: 0.1251 - accuracy: 0.9509 - val_loss: 0.4738 - val_accuracy: 0.8925 - lr: 1.2500e-04
Epoch 23/25
121/121 [==============================] - 19s 157ms/step - loss: 0.1145 - accuracy: 0.9561 - val_loss: 0.4030 - val_accuracy: 0.9042 - lr: 6.2500e-05
Epoch 24/25
121/121 [==============================] - ETA: 0s - loss: 0.1121 - accuracy: 0.9587
Epoch 24: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
121/121 [==============================] - 18s 151ms/step - loss: 0.1121 - accuracy: 0.9587 - val_loss: 0.4251 - val_accuracy: 0.9065 - lr: 6.2500e-05
Epoch 25/25
121/121 [==============================] - 19s 156ms/step - loss: 0.0963 - accuracy: 0.9665 - val_loss: 0.4347 - val_accuracy: 0.9089 - lr: 3.1250e-05

Model Evaluation

plt.plot(lron_da_history.history['accuracy'])
plt.plot(lron_da_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations

• We still see signs of the training data overfitting and oscillations of the validation data until around epoch 16.
• Training and validation are still pretty divergent.

plt.plot(lron_da_history.history['loss'])
plt.plot(lron_da_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Loss flattens out beyond epoch 10.

Evaluate the model on test data

lron_da_test_accuracy = lron_da_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 66ms/step - loss: 0.4274 - accuracy: 0.8947

best_model_accuracy = lron_da_history.history['accuracy'][np.argmin(lron_da_history.history['loss'])]
best_model_accuracy

0.9664673805236816

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da_test_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.966467
Name: Complex Model with batch normalization, learning rate reduction and basic data augmentation, dtype: float64
accuracy    0.894737
Name: Complex Model with batch normalization, learning rate reduction and basic data augmentation, dtype: float64

Observations

• Test Accuracy is 89%, so a slight improvement
• Model accuracy with least loss is 97%, so overfitting a bit less.

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=lron_da_model.predict(X_test_normalized)

15/15 [==============================] - 1s 63ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• Black-grass has improved from 8 to 3 misindentified
• Common Chickweed, Common Wheat, Fat Hen, Scentless Mayweed have all remained a low percentage of misidentifications.
• Maize has been 100% identified correctly.
• Sheperds Purse has increased by 1 in misidentification.
• Loose Silky-bent is now the largest misclassification.
• Most others remain about the same level of misclassification.

Model adjustment by adding steps per epoch

We will add steps per epoch to set how many times we get a new batch from the generator to try and improve the model. This will end up discarding 7 images in training. This will work in conjunction with ReduceLROnPlateau to better decrease the learning rate.

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
                              rotation_range=20,
                              fill_mode='nearest'
                              )

model_name = "Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch"

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
lron_da_spe_model = Sequential()

# create convolution and max-pooling layers with input size of images
lron_da_spe_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da_spe_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
lron_da_spe_model.add(Flatten())

# add a fully connected layer
lron_da_spe_model.add(Dense(128, activation='relu'))
lron_da_spe_model.add(Dropout(0.3))

# add a fully connected layer
lron_da_spe_model.add(Dense(512, activation='relu'))
lron_da_spe_model.add(Dropout(0.3))

# add a fully connected layer
lron_da_spe_model.add(Dense(256, activation='relu'))
lron_da_spe_model.add(Dropout(0.3))

# output layer with 12 neurons and softmax
lron_da_spe_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()
# Compile the model
lron_da_spe_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
lron_da_spe_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 batch_normalization (Batch  (None, 64, 64, 32)        128       
 Normalization)                                                  
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 batch_normalization_1 (Bat  (None, 32, 32, 64)        256       
 chNormalization)                                                
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 batch_normalization_2 (Bat  (None, 16, 16, 128)       512       
 chNormalization)                                                
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 batch_normalization_3 (Bat  (None, 8, 8, 128)         512       
 chNormalization)                                                
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________

Fitting the model on the train data

lron_da_spe_history = lron_da_spe_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       shuffle=False),
                                       epochs=epochs,
                                       steps_per_epoch=X_train_normalized.shape[0] // batch_size, #errors out of range
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,callbacks=[learning_rate_reduction])

Epoch 1/25
120/120 [==============================] - 20s 151ms/step - loss: 2.0092 - accuracy: 0.3166 - val_loss: 8.3740 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 2/25
120/120 [==============================] - 19s 155ms/step - loss: 1.3914 - accuracy: 0.5266 - val_loss: 11.3857 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 3/25
120/120 [==============================] - ETA: 0s - loss: 1.1237 - accuracy: 0.6275
Epoch 3: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
120/120 [==============================] - 19s 157ms/step - loss: 1.1237 - accuracy: 0.6275 - val_loss: 15.8686 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 4/25
120/120 [==============================] - 19s 162ms/step - loss: 0.8396 - accuracy: 0.7132 - val_loss: 6.4671 - val_accuracy: 0.1565 - lr: 5.0000e-04
Epoch 5/25
120/120 [==============================] - 19s 162ms/step - loss: 0.6952 - accuracy: 0.7507 - val_loss: 2.5973 - val_accuracy: 0.3879 - lr: 5.0000e-04
Epoch 6/25
120/120 [==============================] - 19s 159ms/step - loss: 0.6119 - accuracy: 0.7832 - val_loss: 0.7144 - val_accuracy: 0.7710 - lr: 5.0000e-04
Epoch 7/25
120/120 [==============================] - 19s 157ms/step - loss: 0.5156 - accuracy: 0.8144 - val_loss: 0.5888 - val_accuracy: 0.8248 - lr: 5.0000e-04
Epoch 8/25
120/120 [==============================] - 18s 150ms/step - loss: 0.4656 - accuracy: 0.8391 - val_loss: 0.7592 - val_accuracy: 0.7850 - lr: 5.0000e-04
Epoch 9/25
120/120 [==============================] - ETA: 0s - loss: 0.4195 - accuracy: 0.8474
Epoch 9: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
120/120 [==============================] - 18s 149ms/step - loss: 0.4195 - accuracy: 0.8474 - val_loss: 0.9602 - val_accuracy: 0.7243 - lr: 5.0000e-04
Epoch 10/25
120/120 [==============================] - 18s 153ms/step - loss: 0.3632 - accuracy: 0.8708 - val_loss: 0.4819 - val_accuracy: 0.8621 - lr: 2.5000e-04
Epoch 11/25
120/120 [==============================] - 18s 152ms/step - loss: 0.2995 - accuracy: 0.8852 - val_loss: 0.4726 - val_accuracy: 0.8528 - lr: 2.5000e-04
Epoch 12/25
120/120 [==============================] - ETA: 0s - loss: 0.2853 - accuracy: 0.8954
Epoch 12: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
120/120 [==============================] - 19s 154ms/step - loss: 0.2853 - accuracy: 0.8954 - val_loss: 0.4542 - val_accuracy: 0.8598 - lr: 2.5000e-04
Epoch 13/25
120/120 [==============================] - 19s 159ms/step - loss: 0.2356 - accuracy: 0.9119 - val_loss: 0.4342 - val_accuracy: 0.8785 - lr: 1.2500e-04
Epoch 14/25
120/120 [==============================] - 19s 157ms/step - loss: 0.2166 - accuracy: 0.9130 - val_loss: 0.4105 - val_accuracy: 0.8925 - lr: 1.2500e-04
Epoch 15/25
120/120 [==============================] - 18s 150ms/step - loss: 0.1997 - accuracy: 0.9263 - val_loss: 0.4178 - val_accuracy: 0.8902 - lr: 1.2500e-04
Epoch 16/25
120/120 [==============================] - ETA: 0s - loss: 0.1965 - accuracy: 0.9266
Epoch 16: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
120/120 [==============================] - 18s 153ms/step - loss: 0.1965 - accuracy: 0.9266 - val_loss: 0.4869 - val_accuracy: 0.8879 - lr: 1.2500e-04
Epoch 17/25
120/120 [==============================] - 18s 146ms/step - loss: 0.1731 - accuracy: 0.9342 - val_loss: 0.4373 - val_accuracy: 0.8902 - lr: 6.2500e-05
Epoch 18/25
120/120 [==============================] - ETA: 0s - loss: 0.1698 - accuracy: 0.9355
Epoch 18: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
120/120 [==============================] - 17s 140ms/step - loss: 0.1698 - accuracy: 0.9355 - val_loss: 0.4558 - val_accuracy: 0.8715 - lr: 6.2500e-05
Epoch 19/25
120/120 [==============================] - 17s 141ms/step - loss: 0.1718 - accuracy: 0.9389 - val_loss: 0.4159 - val_accuracy: 0.8972 - lr: 3.1250e-05
Epoch 20/25
120/120 [==============================] - 19s 156ms/step - loss: 0.1567 - accuracy: 0.9434 - val_loss: 0.4377 - val_accuracy: 0.8832 - lr: 3.1250e-05
Epoch 21/25
120/120 [==============================] - 19s 156ms/step - loss: 0.1580 - accuracy: 0.9410 - val_loss: 0.4192 - val_accuracy: 0.8995 - lr: 3.1250e-05
Epoch 22/25
120/120 [==============================] - 18s 148ms/step - loss: 0.1533 - accuracy: 0.9405 - val_loss: 0.4188 - val_accuracy: 0.8925 - lr: 3.1250e-05
Epoch 23/25
120/120 [==============================] - 19s 159ms/step - loss: 0.1452 - accuracy: 0.9447 - val_loss: 0.4233 - val_accuracy: 0.9042 - lr: 3.1250e-05
Epoch 24/25
120/120 [==============================] - 19s 158ms/step - loss: 0.1479 - accuracy: 0.9520 - val_loss: 0.4453 - val_accuracy: 0.8738 - lr: 3.1250e-05
Epoch 25/25
120/120 [==============================] - ETA: 0s - loss: 0.1462 - accuracy: 0.9473
Epoch 25: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
120/120 [==============================] - 19s 159ms/step - loss: 0.1462 - accuracy: 0.9473 - val_loss: 0.4296 - val_accuracy: 0.8972 - lr: 3.1250e-05

Model Evaluation

plt.plot(lron_da_spe_history.history['accuracy'])
plt.plot(lron_da_spe_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations

• We still see signs of the training data overfitting though the validation data is much closer.
• Training and validation are still pretty divergent.

plt.plot(lron_da_spe_history.history['loss'])
plt.plot(lron_da_spe_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations

• Loss flattens around epoch 5.

Evaluate the model on test data

lron_da_spe_test_accuracy = lron_da_spe_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 65ms/step - loss: 0.4318 - accuracy: 0.8968

best_model_accuracy = lron_da_spe_history.history['accuracy'][np.argmin(lron_da_spe_history.history['loss'])]
best_model_accuracy

0.9446920156478882

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da_spe_test_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.944692
Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch, dtype: float64
accuracy    0.896842
Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch, dtype: float64

Observations

• Test Accuracy is 90%
• Training model accuracy with least loss is 95%

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=lron_da_spe_model.predict(X_test_normalized)

15/15 [==============================] - 1s 66ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• Misclassification of Black-grass has worsened
• Misclassification of Loose Silky-bent has improved
• Most others remain about the same level of misclassification.

Increase the epochs and add early stopping to limit learning

In this model we will increase the epochs to try and improve learning, but limit it using the early stopping callback to prevent too many epochs from running.

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

model_name = "Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping"

# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
                              rotation_range=20,
                              fill_mode='nearest'
                              )

early_stopping__callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
lron_da2_spe_es_model = Sequential()

# create convolution and max-pooling layers with input size of images
lron_da2_spe_es_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da2_spe_es_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da2_spe_es_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))

# add BatchNormalization
lron_da2_spe_es_model.add(BatchNormalization())

# create convolution and max-pooling layers
lron_da2_spe_es_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
lron_da2_spe_es_model.add(Flatten())

# add a fully connected layer
lron_da2_spe_es_model.add(Dense(128, activation='relu'))
lron_da2_spe_es_model.add(Dropout(0.3))

# add a fully connected layer
lron_da2_spe_es_model.add(Dense(512, activation='relu'))
lron_da2_spe_es_model.add(Dropout(0.3))

# add a fully connected layer
lron_da2_spe_es_model.add(Dense(256, activation='relu'))
lron_da2_spe_es_model.add(Dropout(0.3))

# output layer with 12 neurons and softmax
lron_da2_spe_es_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
lron_da2_spe_es_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
lron_da2_spe_es_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 8, 8, 128)         512       
 Normalization)                                                  
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 704076 (2.69 MB)
Trainable params: 703820 (2.68 MB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________

Fitting the model on the train data

lron_da2_spe_es_history = lron_da2_spe_es_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       shuffle=False),
                                       epochs=100,
                                       steps_per_epoch=X_train_normalized.shape[0] // batch_size, #errors out of range
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,
                                       callbacks=[learning_rate_reduction,early_stopping__callback])

Epoch 1/100
120/120 [==============================] - 14s 105ms/step - loss: 2.0579 - accuracy: 0.2734 - val_loss: 2.4149 - val_accuracy: 0.1893 - lr: 0.0010
Epoch 2/100
120/120 [==============================] - 12s 97ms/step - loss: 1.4825 - accuracy: 0.4747 - val_loss: 2.2907 - val_accuracy: 0.2103 - lr: 0.0010
Epoch 3/100
120/120 [==============================] - 12s 97ms/step - loss: 1.3135 - accuracy: 0.5389 - val_loss: 1.8938 - val_accuracy: 0.5561 - lr: 0.0010
Epoch 4/100
120/120 [==============================] - 12s 98ms/step - loss: 1.1223 - accuracy: 0.6068 - val_loss: 1.5954 - val_accuracy: 0.4089 - lr: 0.0010
Epoch 5/100
120/120 [==============================] - 12s 98ms/step - loss: 0.9872 - accuracy: 0.6569 - val_loss: 1.2421 - val_accuracy: 0.6542 - lr: 0.0010
Epoch 6/100
120/120 [==============================] - 11s 95ms/step - loss: 0.9010 - accuracy: 0.6865 - val_loss: 0.7893 - val_accuracy: 0.7570 - lr: 0.0010
Epoch 7/100
120/120 [==============================] - 11s 94ms/step - loss: 0.8140 - accuracy: 0.7145 - val_loss: 1.7838 - val_accuracy: 0.3902 - lr: 0.0010
Epoch 8/100
120/120 [==============================] - ETA: 0s - loss: 0.7933 - accuracy: 0.7261
Epoch 8: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
120/120 [==============================] - 11s 92ms/step - loss: 0.7933 - accuracy: 0.7261 - val_loss: 0.7397 - val_accuracy: 0.7523 - lr: 0.0010
Epoch 9/100
120/120 [==============================] - 11s 93ms/step - loss: 0.5773 - accuracy: 0.7940 - val_loss: 0.9857 - val_accuracy: 0.7173 - lr: 5.0000e-04
Epoch 10/100
120/120 [==============================] - ETA: 0s - loss: 0.5624 - accuracy: 0.8042
Epoch 10: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
120/120 [==============================] - 11s 89ms/step - loss: 0.5624 - accuracy: 0.8042 - val_loss: 0.7674 - val_accuracy: 0.7266 - lr: 5.0000e-04
Epoch 11/100
120/120 [==============================] - 10s 84ms/step - loss: 0.4956 - accuracy: 0.8218 - val_loss: 0.5210 - val_accuracy: 0.8318 - lr: 2.5000e-04
Epoch 12/100
120/120 [==============================] - 11s 93ms/step - loss: 0.4709 - accuracy: 0.8320 - val_loss: 0.5778 - val_accuracy: 0.8178 - lr: 2.5000e-04
Epoch 13/100
120/120 [==============================] - 11s 94ms/step - loss: 0.4226 - accuracy: 0.8503 - val_loss: 0.5153 - val_accuracy: 0.8458 - lr: 2.5000e-04
Epoch 14/100
120/120 [==============================] - 11s 93ms/step - loss: 0.4193 - accuracy: 0.8574 - val_loss: 0.5674 - val_accuracy: 0.8154 - lr: 2.5000e-04
Epoch 15/100
120/120 [==============================] - 11s 94ms/step - loss: 0.4212 - accuracy: 0.8524 - val_loss: 0.5017 - val_accuracy: 0.8668 - lr: 2.5000e-04
Epoch 16/100
120/120 [==============================] - 11s 95ms/step - loss: 0.3924 - accuracy: 0.8595 - val_loss: 0.5353 - val_accuracy: 0.8341 - lr: 2.5000e-04
Epoch 17/100
120/120 [==============================] - ETA: 0s - loss: 0.3769 - accuracy: 0.8624
Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
120/120 [==============================] - 11s 94ms/step - loss: 0.3769 - accuracy: 0.8624 - val_loss: 0.5655 - val_accuracy: 0.8341 - lr: 2.5000e-04
Epoch 18/100
120/120 [==============================] - 11s 91ms/step - loss: 0.3533 - accuracy: 0.8700 - val_loss: 0.5138 - val_accuracy: 0.8598 - lr: 1.2500e-04
Epoch 19/100
120/120 [==============================] - ETA: 0s - loss: 0.3327 - accuracy: 0.8765
Epoch 19: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
120/120 [==============================] - 11s 92ms/step - loss: 0.3327 - accuracy: 0.8765 - val_loss: 0.5110 - val_accuracy: 0.8481 - lr: 1.2500e-04
Epoch 20/100
120/120 [==============================] - 11s 92ms/step - loss: 0.3215 - accuracy: 0.8802 - val_loss: 0.4936 - val_accuracy: 0.8621 - lr: 6.2500e-05
Epoch 21/100
120/120 [==============================] - ETA: 0s - loss: 0.3167 - accuracy: 0.8841
Epoch 21: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
120/120 [==============================] - 11s 92ms/step - loss: 0.3167 - accuracy: 0.8841 - val_loss: 0.4775 - val_accuracy: 0.8598 - lr: 6.2500e-05
Epoch 22/100
120/120 [==============================] - 11s 95ms/step - loss: 0.3097 - accuracy: 0.8779 - val_loss: 0.4919 - val_accuracy: 0.8551 - lr: 3.1250e-05
Epoch 23/100
120/120 [==============================] - ETA: 0s - loss: 0.2976 - accuracy: 0.8805
Epoch 23: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
120/120 [==============================] - 11s 95ms/step - loss: 0.2976 - accuracy: 0.8805 - val_loss: 0.5168 - val_accuracy: 0.8575 - lr: 3.1250e-05
Epoch 24/100
120/120 [==============================] - 11s 87ms/step - loss: 0.3017 - accuracy: 0.8849 - val_loss: 0.5004 - val_accuracy: 0.8528 - lr: 1.5625e-05
Epoch 25/100
120/120 [==============================] - ETA: 0s - loss: 0.2963 - accuracy: 0.8907
Epoch 25: ReduceLROnPlateau reducing learning rate to 1e-05.
120/120 [==============================] - 11s 88ms/step - loss: 0.2963 - accuracy: 0.8907 - val_loss: 0.4963 - val_accuracy: 0.8481 - lr: 1.5625e-05
Epoch 26/100
120/120 [==============================] - 11s 92ms/step - loss: 0.2998 - accuracy: 0.8904 - val_loss: 0.4889 - val_accuracy: 0.8621 - lr: 1.0000e-05
Epoch 27/100
120/120 [==============================] - 11s 92ms/step - loss: 0.2997 - accuracy: 0.8920 - val_loss: 0.4910 - val_accuracy: 0.8621 - lr: 1.0000e-05
Epoch 28/100
120/120 [==============================] - 11s 91ms/step - loss: 0.3003 - accuracy: 0.8883 - val_loss: 0.4845 - val_accuracy: 0.8621 - lr: 1.0000e-05
Epoch 29/100
120/120 [==============================] - 11s 94ms/step - loss: 0.3036 - accuracy: 0.8891 - val_loss: 0.4937 - val_accuracy: 0.8645 - lr: 1.0000e-05
Epoch 30/100
120/120 [==============================] - 11s 95ms/step - loss: 0.2872 - accuracy: 0.8946 - val_loss: 0.4926 - val_accuracy: 0.8645 - lr: 1.0000e-05
Epoch 31/100
120/120 [==============================] - 11s 94ms/step - loss: 0.2984 - accuracy: 0.8834 - val_loss: 0.4885 - val_accuracy: 0.8621 - lr: 1.0000e-05
Epoch 32/100
120/120 [==============================] - 11s 95ms/step - loss: 0.2692 - accuracy: 0.9009 - val_loss: 0.4908 - val_accuracy: 0.8598 - lr: 1.0000e-05
Epoch 33/100
120/120 [==============================] - 11s 91ms/step - loss: 0.2913 - accuracy: 0.8923 - val_loss: 0.4871 - val_accuracy: 0.8528 - lr: 1.0000e-05
Epoch 34/100
120/120 [==============================] - 11s 90ms/step - loss: 0.2950 - accuracy: 0.8886 - val_loss: 0.4895 - val_accuracy: 0.8645 - lr: 1.0000e-05
Epoch 35/100
120/120 [==============================] - 11s 92ms/step - loss: 0.2759 - accuracy: 0.8949 - val_loss: 0.5009 - val_accuracy: 0.8621 - lr: 1.0000e-05

Model Evaluation

plt.plot(lron_da2_spe_es_history.history['accuracy'])
plt.plot(lron_da2_spe_es_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• From the graph we can see that the training accuracy and validation are reasonably close once epoch 18 is reached.
• this model has the closest trianing and validation we have seen so far.

plt.plot(lron_da2_spe_es_history.history['loss'])
plt.plot(lron_da2_spe_es_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Loss flattens after epoch 15. I suspect if w elimited this model to 15 epochs we might se very close results.

Evaluate the model on test data

lron_da2_spe_es_accuracy = lron_da2_spe_es_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 47ms/step - loss: 0.5379 - accuracy: 0.8337

best_model_accuracy = lron_da2_spe_es_history.history['accuracy'][np.argmin(lron_da2_spe_es_history.history['loss'])]
best_model_accuracy

0.9009174108505249

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da2_spe_es_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.900917
Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping, dtype: float64
accuracy    0.833684
Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping, dtype: float64

Observations

• Test Accuracy is 83%
• Training model accuracy with least loss is 90%

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=lron_da2_spe_es_model.predict(X_test_normalized)    #<---- check this is right for others

15/15 [==============================] - 1s 50ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• Misclassification of Black-grass has worsened.
• Misclassification of Loose Silky-bent has improved.
• Most others have increased level of misclassification.

Add more complex data augmentaion to model

In this model we will try using more data augmentaion to improve the model. We will keep everything else the same.

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

train_datagen = ImageDataGenerator(
        #rescale=1./255,
        shear_range = 0.2,
        fill_mode='nearest',
        rotation_range=180,  # randomly rotate images in the range (was 180)
        zoom_range = 0.2, # Randomly zoom image (was.1 with others)
        width_shift_range=0.2,  # randomly shift images horizontally
        height_shift_range=0.2,  # randomly shift images vertically
        horizontal_flip=True,  # randomly flip images horizontally
        vertical_flip=True  # randomly flip images vertically
    )

early_stopping__callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)

model_name = "Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping"

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)

# Intializing a sequential model
lron_da2_spe_model = Sequential()

# create convolution and max-pooling layers with input size of images
lron_da2_spe_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da2_spe_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da2_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# create convolution and max-pooling layers
lron_da2_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))

# flatten the output of the conv layer
lron_da2_spe_model.add(Flatten())

# add a fully connected layer
lron_da2_spe_model.add(Dense(128, activation='relu'))
lron_da2_spe_model.add(Dropout(0.3))

# add a fully connected layer
lron_da2_spe_model.add(Dense(512, activation='relu'))
lron_da2_spe_model.add(Dropout(0.3))

# add a fully connected layer
lron_da2_spe_model.add(Dense(256, activation='relu'))
lron_da2_spe_model.add(Dropout(0.3))

# output layer with 12 neurons and softmax
lron_da2_spe_model.add(Dense(12, activation='softmax'))

# Adam Optimizer
opt=Adam()

# Compile the model
lron_da2_spe_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# generate the summary
lron_da2_spe_model.summary()

width:   64
height:  64
channel: 3
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 64, 64, 32)        896       
                                                                 
 batch_normalization (Batch  (None, 64, 64, 32)        128       
 Normalization)                                                  
                                                                 
 max_pooling2d (MaxPooling2  (None, 32, 32, 32)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 batch_normalization_1 (Bat  (None, 32, 32, 64)        256       
 chNormalization)                                                
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 16, 16, 64)        0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 batch_normalization_2 (Bat  (None, 16, 16, 128)       512       
 chNormalization)                                                
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 8, 8, 128)         0         
 g2D)                                                            
                                                                 
 conv2d_3 (Conv2D)           (None, 8, 8, 128)         147584    
                                                                 
 batch_normalization_3 (Bat  (None, 8, 8, 128)         512       
 chNormalization)                                                
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 4, 4, 128)         0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________

Fitting the model on the train data

lron_da2_spe_history = lron_da2_spe_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       shuffle=False),
                                       epochs=100,
                                       steps_per_epoch=X_train_normalized.shape[0] // batch_size, #errors out of range
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,callbacks=[learning_rate_reduction,early_stopping__callback])

Epoch 1/100
120/120 [==============================] - 18s 141ms/step - loss: 2.1294 - accuracy: 0.2634 - val_loss: 5.2550 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 2/100
120/120 [==============================] - 19s 154ms/step - loss: 1.6919 - accuracy: 0.4204 - val_loss: 6.8600 - val_accuracy: 0.1379 - lr: 0.0010
Epoch 3/100
120/120 [==============================] - 18s 150ms/step - loss: 1.4913 - accuracy: 0.4750 - val_loss: 3.6939 - val_accuracy: 0.1682 - lr: 0.0010
Epoch 4/100
120/120 [==============================] - 18s 152ms/step - loss: 1.3416 - accuracy: 0.5394 - val_loss: 4.3308 - val_accuracy: 0.1449 - lr: 0.0010
Epoch 5/100
120/120 [==============================] - 18s 149ms/step - loss: 1.2554 - accuracy: 0.5740 - val_loss: 1.7323 - val_accuracy: 0.3762 - lr: 0.0010
Epoch 6/100
120/120 [==============================] - 18s 151ms/step - loss: 1.1122 - accuracy: 0.6176 - val_loss: 1.6744 - val_accuracy: 0.4673 - lr: 0.0010
Epoch 7/100
120/120 [==============================] - 18s 146ms/step - loss: 1.0057 - accuracy: 0.6467 - val_loss: 1.4680 - val_accuracy: 0.5584 - lr: 0.0010
Epoch 8/100
120/120 [==============================] - 18s 150ms/step - loss: 0.9692 - accuracy: 0.6731 - val_loss: 1.2056 - val_accuracy: 0.6168 - lr: 0.0010
Epoch 9/100
120/120 [==============================] - 17s 142ms/step - loss: 0.8942 - accuracy: 0.7062 - val_loss: 0.9388 - val_accuracy: 0.6939 - lr: 0.0010
Epoch 10/100
120/120 [==============================] - 19s 154ms/step - loss: 0.8869 - accuracy: 0.7012 - val_loss: 1.2399 - val_accuracy: 0.6098 - lr: 0.0010
Epoch 11/100
120/120 [==============================] - ETA: 0s - loss: 0.7852 - accuracy: 0.7258
Epoch 11: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
120/120 [==============================] - 18s 148ms/step - loss: 0.7852 - accuracy: 0.7258 - val_loss: 0.9927 - val_accuracy: 0.6846 - lr: 0.0010
Epoch 12/100
120/120 [==============================] - 18s 149ms/step - loss: 0.7010 - accuracy: 0.7554 - val_loss: 0.9238 - val_accuracy: 0.7103 - lr: 5.0000e-04
Epoch 13/100
120/120 [==============================] - 18s 149ms/step - loss: 0.6523 - accuracy: 0.7701 - val_loss: 0.5948 - val_accuracy: 0.8084 - lr: 5.0000e-04
Epoch 14/100
120/120 [==============================] - 19s 158ms/step - loss: 0.6177 - accuracy: 0.7893 - val_loss: 0.5547 - val_accuracy: 0.8294 - lr: 5.0000e-04
Epoch 15/100
120/120 [==============================] - 18s 152ms/step - loss: 0.5863 - accuracy: 0.7937 - val_loss: 0.6298 - val_accuracy: 0.7921 - lr: 5.0000e-04
Epoch 16/100
120/120 [==============================] - 19s 155ms/step - loss: 0.5658 - accuracy: 0.8089 - val_loss: 0.4873 - val_accuracy: 0.8318 - lr: 5.0000e-04
Epoch 17/100
120/120 [==============================] - 17s 140ms/step - loss: 0.5366 - accuracy: 0.8066 - val_loss: 0.7136 - val_accuracy: 0.7593 - lr: 5.0000e-04
Epoch 18/100
120/120 [==============================] - ETA: 0s - loss: 0.5118 - accuracy: 0.8155
Epoch 18: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
120/120 [==============================] - 17s 141ms/step - loss: 0.5118 - accuracy: 0.8155 - val_loss: 1.4669 - val_accuracy: 0.6332 - lr: 5.0000e-04
Epoch 19/100
120/120 [==============================] - 18s 152ms/step - loss: 0.4726 - accuracy: 0.8393 - val_loss: 0.4200 - val_accuracy: 0.8551 - lr: 2.5000e-04
Epoch 20/100
120/120 [==============================] - 19s 162ms/step - loss: 0.4393 - accuracy: 0.8438 - val_loss: 0.3603 - val_accuracy: 0.8785 - lr: 2.5000e-04
Epoch 21/100
120/120 [==============================] - 18s 151ms/step - loss: 0.4371 - accuracy: 0.8474 - val_loss: 0.3736 - val_accuracy: 0.8762 - lr: 2.5000e-04
Epoch 22/100
120/120 [==============================] - ETA: 0s - loss: 0.4297 - accuracy: 0.8459
Epoch 22: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
120/120 [==============================] - 18s 150ms/step - loss: 0.4297 - accuracy: 0.8459 - val_loss: 0.6631 - val_accuracy: 0.7874 - lr: 2.5000e-04
Epoch 23/100
120/120 [==============================] - 19s 154ms/step - loss: 0.3933 - accuracy: 0.8535 - val_loss: 0.3623 - val_accuracy: 0.8925 - lr: 1.2500e-04
Epoch 24/100
120/120 [==============================] - 19s 155ms/step - loss: 0.3791 - accuracy: 0.8553 - val_loss: 0.3262 - val_accuracy: 0.9089 - lr: 1.2500e-04
Epoch 25/100
120/120 [==============================] - 19s 154ms/step - loss: 0.3734 - accuracy: 0.8676 - val_loss: 0.3479 - val_accuracy: 0.8925 - lr: 1.2500e-04
Epoch 26/100
120/120 [==============================] - ETA: 0s - loss: 0.3758 - accuracy: 0.8621
Epoch 26: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
120/120 [==============================] - 18s 152ms/step - loss: 0.3758 - accuracy: 0.8621 - val_loss: 0.4046 - val_accuracy: 0.8575 - lr: 1.2500e-04
Epoch 27/100
120/120 [==============================] - 18s 151ms/step - loss: 0.3534 - accuracy: 0.8747 - val_loss: 0.3166 - val_accuracy: 0.9089 - lr: 6.2500e-05
Epoch 28/100
120/120 [==============================] - 16s 136ms/step - loss: 0.3565 - accuracy: 0.8710 - val_loss: 0.3039 - val_accuracy: 0.9206 - lr: 6.2500e-05
Epoch 29/100
120/120 [==============================] - 18s 149ms/step - loss: 0.3589 - accuracy: 0.8679 - val_loss: 0.3348 - val_accuracy: 0.8925 - lr: 6.2500e-05
Epoch 30/100
120/120 [==============================] - ETA: 0s - loss: 0.3312 - accuracy: 0.8820
Epoch 30: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
120/120 [==============================] - 19s 155ms/step - loss: 0.3312 - accuracy: 0.8820 - val_loss: 0.3366 - val_accuracy: 0.9019 - lr: 6.2500e-05
Epoch 31/100
120/120 [==============================] - 19s 158ms/step - loss: 0.3349 - accuracy: 0.8765 - val_loss: 0.3172 - val_accuracy: 0.9089 - lr: 3.1250e-05
Epoch 32/100
120/120 [==============================] - ETA: 0s - loss: 0.3245 - accuracy: 0.8860
Epoch 32: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
120/120 [==============================] - 18s 150ms/step - loss: 0.3245 - accuracy: 0.8860 - val_loss: 0.3039 - val_accuracy: 0.9019 - lr: 3.1250e-05
Epoch 33/100
120/120 [==============================] - 19s 161ms/step - loss: 0.3343 - accuracy: 0.8826 - val_loss: 0.2923 - val_accuracy: 0.9159 - lr: 1.5625e-05
Epoch 34/100
120/120 [==============================] - 19s 160ms/step - loss: 0.3300 - accuracy: 0.8744 - val_loss: 0.2968 - val_accuracy: 0.9252 - lr: 1.5625e-05
Epoch 35/100
120/120 [==============================] - 18s 152ms/step - loss: 0.3088 - accuracy: 0.8878 - val_loss: 0.2976 - val_accuracy: 0.9112 - lr: 1.5625e-05
Epoch 36/100
120/120 [==============================] - ETA: 0s - loss: 0.3067 - accuracy: 0.8868
Epoch 36: ReduceLROnPlateau reducing learning rate to 1e-05.
120/120 [==============================] - 18s 151ms/step - loss: 0.3067 - accuracy: 0.8868 - val_loss: 0.3018 - val_accuracy: 0.9065 - lr: 1.5625e-05
Epoch 37/100
120/120 [==============================] - 19s 154ms/step - loss: 0.3057 - accuracy: 0.8873 - val_loss: 0.3005 - val_accuracy: 0.9089 - lr: 1.0000e-05
Epoch 38/100
120/120 [==============================] - 19s 156ms/step - loss: 0.3283 - accuracy: 0.8823 - val_loss: 0.3004 - val_accuracy: 0.9206 - lr: 1.0000e-05
Epoch 39/100
120/120 [==============================] - 18s 149ms/step - loss: 0.3083 - accuracy: 0.8915 - val_loss: 0.2974 - val_accuracy: 0.9182 - lr: 1.0000e-05
Epoch 40/100
120/120 [==============================] - 17s 143ms/step - loss: 0.3073 - accuracy: 0.8855 - val_loss: 0.3021 - val_accuracy: 0.9089 - lr: 1.0000e-05
Epoch 41/100
120/120 [==============================] - 16s 132ms/step - loss: 0.3086 - accuracy: 0.8844 - val_loss: 0.3040 - val_accuracy: 0.8995 - lr: 1.0000e-05
Epoch 42/100
120/120 [==============================] - 16s 133ms/step - loss: 0.3187 - accuracy: 0.8789 - val_loss: 0.3080 - val_accuracy: 0.8972 - lr: 1.0000e-05
Epoch 43/100
120/120 [==============================] - 17s 143ms/step - loss: 0.3060 - accuracy: 0.8931 - val_loss: 0.3092 - val_accuracy: 0.8995 - lr: 1.0000e-05
Epoch 44/100
120/120 [==============================] - 18s 149ms/step - loss: 0.3206 - accuracy: 0.8760 - val_loss: 0.3026 - val_accuracy: 0.9206 - lr: 1.0000e-05
Epoch 45/100
120/120 [==============================] - 18s 153ms/step - loss: 0.3113 - accuracy: 0.8865 - val_loss: 0.3064 - val_accuracy: 0.9112 - lr: 1.0000e-05
Epoch 46/100
120/120 [==============================] - 19s 158ms/step - loss: 0.3138 - accuracy: 0.8875 - val_loss: 0.3136 - val_accuracy: 0.8972 - lr: 1.0000e-05
Epoch 47/100
120/120 [==============================] - 19s 157ms/step - loss: 0.3208 - accuracy: 0.8799 - val_loss: 0.3089 - val_accuracy: 0.8995 - lr: 1.0000e-05
Epoch 48/100
120/120 [==============================] - 18s 152ms/step - loss: 0.3110 - accuracy: 0.8855 - val_loss: 0.3066 - val_accuracy: 0.9112 - lr: 1.0000e-05
Epoch 49/100
120/120 [==============================] - 19s 157ms/step - loss: 0.3132 - accuracy: 0.8931 - val_loss: 0.3047 - val_accuracy: 0.9112 - lr: 1.0000e-05
Epoch 50/100
120/120 [==============================] - 18s 150ms/step - loss: 0.3009 - accuracy: 0.8865 - val_loss: 0.3015 - val_accuracy: 0.9136 - lr: 1.0000e-05
Epoch 51/100
120/120 [==============================] - 19s 154ms/step - loss: 0.3149 - accuracy: 0.8899 - val_loss: 0.3044 - val_accuracy: 0.9065 - lr: 1.0000e-05
Epoch 52/100
120/120 [==============================] - 19s 158ms/step - loss: 0.3048 - accuracy: 0.8896 - val_loss: 0.3044 - val_accuracy: 0.9065 - lr: 1.0000e-05
Epoch 53/100
120/120 [==============================] - 18s 154ms/step - loss: 0.3092 - accuracy: 0.8883 - val_loss: 0.3014 - val_accuracy: 0.9089 - lr: 1.0000e-05
Epoch 54/100
120/120 [==============================] - 18s 150ms/step - loss: 0.3041 - accuracy: 0.8839 - val_loss: 0.3031 - val_accuracy: 0.9089 - lr: 1.0000e-05

Model Evaluation

plt.plot(lron_da2_spe_history.history['accuracy'])
plt.plot(lron_da2_spe_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• From the graph we can see that the training and the validation accuracy was very close.
• Validation accuracy was higher than training in this model.

plt.plot(lron_da2_spe_history.history['loss'])
plt.plot(lron_da2_spe_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• Loss for training and accuracy were lockstep after epoch 22.

Evaluate the model on test data

lron_da2_spe_test_accuracy = lron_da2_spe_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 62ms/step - loss: 0.3626 - accuracy: 0.8758

best_model_accuracy = lron_da2_spe_history.history['accuracy'][np.argmin(lron_da2_spe_history.history['loss'])]
best_model_accuracy

0.8865006566047668

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da2_spe_test_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.886501
Name: Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping, dtype: float64
accuracy    0.875789
Name: Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping, dtype: float64

Observations

• Test Accuracy is 89%
• Training model accuracy with least loss is 88%
• This is our closest model thus far, but we do have models that have better accuracy.

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=lron_da2_spe_model.predict(X_test_normalized)

15/15 [==============================] - 1s 61ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• Misclassification of Black-grass and Loose Silky-bent has worsened
• Most others have varied improvement/worsening of misclassification.
• This model is still not suitable.

Model using VGG16 pre-trained model

In this model we will use transfer learning from the VGG16 model to try and improve our classification. We will only take the bottom layers and of course make VGG16 non-trainable. We will add our own dense layers.

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

model_name = "VGG16 Model"

early_stopping__callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)

from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)


vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (h,w,c))
vgg_model.summary()

width:   64
height:  64
channel: 3
Model: "vgg16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 64, 64, 3)]       0         
                                                                 
 block1_conv1 (Conv2D)       (None, 64, 64, 64)        1792      
                                                                 
 block1_conv2 (Conv2D)       (None, 64, 64, 64)        36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, 32, 32, 64)        0         
                                                                 
 block2_conv1 (Conv2D)       (None, 32, 32, 128)       73856     
                                                                 
 block2_conv2 (Conv2D)       (None, 32, 32, 128)       147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, 16, 16, 128)       0         
                                                                 
 block3_conv1 (Conv2D)       (None, 16, 16, 256)       295168    
                                                                 
 block3_conv2 (Conv2D)       (None, 16, 16, 256)       590080    
                                                                 
 block3_conv3 (Conv2D)       (None, 16, 16, 256)       590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, 8, 8, 256)         0         
                                                                 
 block4_conv1 (Conv2D)       (None, 8, 8, 512)         1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, 4, 4, 512)         0         
                                                                 
 block5_conv1 (Conv2D)       (None, 4, 4, 512)         2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, 4, 4, 512)         2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, 4, 4, 512)         2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, 2, 2, 512)         0         
                                                                 
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
    layer.trainable = False

vgg_new_model = Sequential()

# Adding the convolutional part of the VGG16 model from above
vgg_new_model.add(vgg_model)

# Flattening the output of the VGG16 model because it is from a convolutional layer
vgg_new_model.add(Flatten())

# Adding a fully connected dense layer with 16 neurons
vgg_new_model.add(Dense(128, activation='relu'))
vgg_new_model.add(Dropout(0.3))

# Adding a fully connected dense layer with 16 neurons
vgg_new_model.add(Dense(512, activation='relu'))
vgg_new_model.add(Dropout(0.3))

vgg_new_model.add(Dense(256, activation='relu'))
vgg_new_model.add(Dropout(0.3))

# output layer with 12 neurons 
vgg_new_model.add(Dense(12, activation='softmax'))

# initialize Adam Optimimzer
opt=Adam()
adam_opt = optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)

# Compile model
vgg_new_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
vgg_new_model.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 vgg16 (Functional)          (None, 2, 2, 512)         14714688  
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 dense (Dense)               (None, 128)               262272    
                                                                 
 dropout (Dropout)           (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 512)               66048     
                                                                 
 dropout_1 (Dropout)         (None, 512)               0         
                                                                 
 dense_2 (Dense)             (None, 256)               131328    
                                                                 
 dropout_2 (Dropout)         (None, 256)               0         
                                                                 
 dense_3 (Dense)             (None, 12)                3084      
                                                                 
=================================================================
Total params: 15177420 (57.90 MB)
Trainable params: 462732 (1.77 MB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________

history_vgg16 = vgg_new_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=32,
                                       seed=42,
                                       shuffle=False),
                                       epochs=100,
                                       steps_per_epoch=math.floor(X_train_normalized.shape[0] / batch_size), #error again
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,callbacks=[learning_rate_reduction,early_stopping__callback])

Epoch 1/100
120/120 [==============================] - 24s 195ms/step - loss: 2.2127 - accuracy: 0.2173 - val_loss: 1.9076 - val_accuracy: 0.3294 - lr: 0.0010
Epoch 2/100
120/120 [==============================] - 23s 190ms/step - loss: 1.8482 - accuracy: 0.3394 - val_loss: 1.7546 - val_accuracy: 0.3785 - lr: 0.0010
Epoch 3/100
120/120 [==============================] - 23s 191ms/step - loss: 1.7462 - accuracy: 0.3641 - val_loss: 1.6430 - val_accuracy: 0.3925 - lr: 0.0010
Epoch 4/100
120/120 [==============================] - 23s 191ms/step - loss: 1.6884 - accuracy: 0.3872 - val_loss: 1.5719 - val_accuracy: 0.4299 - lr: 0.0010
Epoch 5/100
120/120 [==============================] - 22s 186ms/step - loss: 1.6371 - accuracy: 0.4128 - val_loss: 1.5323 - val_accuracy: 0.4673 - lr: 0.0010
Epoch 6/100
120/120 [==============================] - 23s 196ms/step - loss: 1.6326 - accuracy: 0.4121 - val_loss: 1.5059 - val_accuracy: 0.4720 - lr: 0.0010
Epoch 7/100
120/120 [==============================] - 23s 192ms/step - loss: 1.5733 - accuracy: 0.4406 - val_loss: 1.5034 - val_accuracy: 0.4346 - lr: 0.0010
Epoch 8/100
120/120 [==============================] - ETA: 0s - loss: 1.5758 - accuracy: 0.4333
Epoch 8: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
120/120 [==============================] - 24s 203ms/step - loss: 1.5758 - accuracy: 0.4333 - val_loss: 1.4820 - val_accuracy: 0.4579 - lr: 0.0010
Epoch 9/100
120/120 [==============================] - 24s 202ms/step - loss: 1.4924 - accuracy: 0.4658 - val_loss: 1.4744 - val_accuracy: 0.4696 - lr: 5.0000e-04
Epoch 10/100
120/120 [==============================] - 24s 202ms/step - loss: 1.4700 - accuracy: 0.4820 - val_loss: 1.3961 - val_accuracy: 0.4953 - lr: 5.0000e-04
Epoch 11/100
120/120 [==============================] - 24s 196ms/step - loss: 1.4724 - accuracy: 0.4755 - val_loss: 1.3956 - val_accuracy: 0.5117 - lr: 5.0000e-04
Epoch 12/100
120/120 [==============================] - 23s 193ms/step - loss: 1.4667 - accuracy: 0.4713 - val_loss: 1.3596 - val_accuracy: 0.5187 - lr: 5.0000e-04
Epoch 13/100
120/120 [==============================] - 24s 199ms/step - loss: 1.4327 - accuracy: 0.4799 - val_loss: 1.3718 - val_accuracy: 0.4836 - lr: 5.0000e-04
Epoch 14/100
120/120 [==============================] - ETA: 0s - loss: 1.4235 - accuracy: 0.5001
Epoch 14: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
120/120 [==============================] - 23s 191ms/step - loss: 1.4235 - accuracy: 0.5001 - val_loss: 1.3431 - val_accuracy: 0.5047 - lr: 5.0000e-04
Epoch 15/100
120/120 [==============================] - 24s 197ms/step - loss: 1.3934 - accuracy: 0.5054 - val_loss: 1.2918 - val_accuracy: 0.5327 - lr: 2.5000e-04
Epoch 16/100
120/120 [==============================] - 24s 200ms/step - loss: 1.3718 - accuracy: 0.5122 - val_loss: 1.3285 - val_accuracy: 0.5164 - lr: 2.5000e-04
Epoch 17/100
120/120 [==============================] - ETA: 0s - loss: 1.3753 - accuracy: 0.5132
Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
120/120 [==============================] - 23s 193ms/step - loss: 1.3753 - accuracy: 0.5132 - val_loss: 1.3135 - val_accuracy: 0.5304 - lr: 2.5000e-04
Epoch 18/100
120/120 [==============================] - 23s 194ms/step - loss: 1.3599 - accuracy: 0.5098 - val_loss: 1.2818 - val_accuracy: 0.5467 - lr: 1.2500e-04
Epoch 19/100
120/120 [==============================] - 23s 191ms/step - loss: 1.3330 - accuracy: 0.5203 - val_loss: 1.2627 - val_accuracy: 0.5537 - lr: 1.2500e-04
Epoch 20/100
120/120 [==============================] - 24s 203ms/step - loss: 1.3244 - accuracy: 0.5274 - val_loss: 1.2748 - val_accuracy: 0.5421 - lr: 1.2500e-04
Epoch 21/100
120/120 [==============================] - 24s 197ms/step - loss: 1.3323 - accuracy: 0.5250 - val_loss: 1.2601 - val_accuracy: 0.5654 - lr: 1.2500e-04
Epoch 22/100
120/120 [==============================] - 23s 193ms/step - loss: 1.3297 - accuracy: 0.5177 - val_loss: 1.2642 - val_accuracy: 0.5444 - lr: 1.2500e-04
Epoch 23/100
120/120 [==============================] - ETA: 0s - loss: 1.3366 - accuracy: 0.5190
Epoch 23: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
120/120 [==============================] - 25s 208ms/step - loss: 1.3366 - accuracy: 0.5190 - val_loss: 1.2741 - val_accuracy: 0.5327 - lr: 1.2500e-04
Epoch 24/100
120/120 [==============================] - 25s 210ms/step - loss: 1.3334 - accuracy: 0.5208 - val_loss: 1.2652 - val_accuracy: 0.5444 - lr: 6.2500e-05
Epoch 25/100
120/120 [==============================] - ETA: 0s - loss: 1.3295 - accuracy: 0.5245
Epoch 25: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
120/120 [==============================] - 26s 212ms/step - loss: 1.3295 - accuracy: 0.5245 - val_loss: 1.2524 - val_accuracy: 0.5561 - lr: 6.2500e-05
Epoch 26/100
120/120 [==============================] - 24s 200ms/step - loss: 1.3380 - accuracy: 0.5169 - val_loss: 1.2521 - val_accuracy: 0.5537 - lr: 3.1250e-05
Epoch 27/100
120/120 [==============================] - ETA: 0s - loss: 1.3091 - accuracy: 0.5376
Epoch 27: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
120/120 [==============================] - 23s 194ms/step - loss: 1.3091 - accuracy: 0.5376 - val_loss: 1.2456 - val_accuracy: 0.5537 - lr: 3.1250e-05
Epoch 28/100
120/120 [==============================] - 24s 201ms/step - loss: 1.2979 - accuracy: 0.5368 - val_loss: 1.2439 - val_accuracy: 0.5584 - lr: 1.5625e-05
Epoch 29/100
120/120 [==============================] - ETA: 0s - loss: 1.3247 - accuracy: 0.5198
Epoch 29: ReduceLROnPlateau reducing learning rate to 1e-05.
120/120 [==============================] - 24s 200ms/step - loss: 1.3247 - accuracy: 0.5198 - val_loss: 1.2445 - val_accuracy: 0.5561 - lr: 1.5625e-05
Epoch 30/100
120/120 [==============================] - 25s 207ms/step - loss: 1.3144 - accuracy: 0.5263 - val_loss: 1.2426 - val_accuracy: 0.5561 - lr: 1.0000e-05
Epoch 31/100
120/120 [==============================] - 24s 200ms/step - loss: 1.3043 - accuracy: 0.5400 - val_loss: 1.2438 - val_accuracy: 0.5491 - lr: 1.0000e-05
Epoch 32/100
120/120 [==============================] - 24s 201ms/step - loss: 1.3245 - accuracy: 0.5174 - val_loss: 1.2439 - val_accuracy: 0.5584 - lr: 1.0000e-05
Epoch 33/100
120/120 [==============================] - 24s 200ms/step - loss: 1.3060 - accuracy: 0.5279 - val_loss: 1.2453 - val_accuracy: 0.5561 - lr: 1.0000e-05
Epoch 34/100
120/120 [==============================] - 23s 194ms/step - loss: 1.3125 - accuracy: 0.5347 - val_loss: 1.2475 - val_accuracy: 0.5537 - lr: 1.0000e-05
Epoch 35/100
120/120 [==============================] - 23s 194ms/step - loss: 1.3189 - accuracy: 0.5261 - val_loss: 1.2461 - val_accuracy: 0.5537 - lr: 1.0000e-05
Epoch 36/100
120/120 [==============================] - 24s 199ms/step - loss: 1.3034 - accuracy: 0.5308 - val_loss: 1.2456 - val_accuracy: 0.5561 - lr: 1.0000e-05
Epoch 37/100
120/120 [==============================] - 25s 212ms/step - loss: 1.3280 - accuracy: 0.5316 - val_loss: 1.2453 - val_accuracy: 0.5561 - lr: 1.0000e-05
Epoch 38/100
120/120 [==============================] - 24s 196ms/step - loss: 1.3096 - accuracy: 0.5326 - val_loss: 1.2452 - val_accuracy: 0.5561 - lr: 1.0000e-05
Epoch 39/100
120/120 [==============================] - 25s 206ms/step - loss: 1.2940 - accuracy: 0.5457 - val_loss: 1.2454 - val_accuracy: 0.5537 - lr: 1.0000e-05
Epoch 40/100
120/120 [==============================] - 25s 208ms/step - loss: 1.2821 - accuracy: 0.5371 - val_loss: 1.2417 - val_accuracy: 0.5561 - lr: 1.0000e-05
Epoch 41/100
120/120 [==============================] - 25s 210ms/step - loss: 1.3148 - accuracy: 0.5263 - val_loss: 1.2415 - val_accuracy: 0.5467 - lr: 1.0000e-05

Model Evaluation

plt.plot(history_vgg16.history['accuracy'])
plt.plot(history_vgg16.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Observations:

• This model never achieves even 60% accuracy, and thus is a poor model compared to the other ones we have.

plt.plot(history_vgg16.history['loss'])
plt.plot(history_vgg16.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Evaluate the model on test data

vgg_accuracy = vgg_new_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 3s 175ms/step - loss: 1.2491 - accuracy: 0.5642

best_model_accuracy = history_vgg16.history['accuracy'][np.argmin(history_vgg16.history['loss'])]
best_model_accuracy

0.5370904207229614

#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = vgg_accuracy[1]

print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])

accuracy    0.53709
Name: VGG16 Model, dtype: float64
accuracy    0.564211
Name: VGG16 Model, dtype: float64

Observations

• Test Accuracy is 56%
• Training model accuracy with least loss is 54%
• This model performed poorly, it may be too generalized

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=vgg_new_model.predict(X_test_normalized)

15/15 [==============================] - 3s 173ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Observations

• This model did not perform well at all, Misclassification of many plants that were even bettwr identified with our basic model. This model may be too generalized or the dense layers are poor.

training_metrics

	accuracy
Base Model	0.700026
Complex Model	0.971146
Complex Model with Class Weights applied	0.939173
Complex Model with Adam optimizer adjusted	0.949051
Complex Model with learning rate reduction and batch normalization	0.985703
Complex Model with batch normalization, learning rate reduction and basic data augmentation	0.966467
Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch	0.944692
Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping	0.900917
Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping	0.886501
VGG16 Model	0.537090

test_metrics

	accuracy
Base Model	0.732632
Complex Model	0.821053
Complex Model with Class Weights applied	0.795789
Complex Model with Adam optimizer adjusted	0.840000
Complex Model with learning rate reduction and batch normalization	0.911579
Complex Model with batch normalization, learning rate reduction and basic data augmentation	0.894737
Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch	0.896842
Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping	0.833684
Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping	0.875789
VGG16 Model	0.564211

Final Model

While the "Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping" was very close on the training and test scorres, for this exercise we want the highest accuracy. that came from model "Complex Model with learning rate reduction and batch normalization" with a 91% accuracy on test data. This will be our final model.

lron_base_model_test_accuracy = lron_base_model.evaluate(X_test_normalized, 
                                                          y_test_encoded, 
                                                          #verbose=2
                                                         )

15/15 [==============================] - 1s 71ms/step - loss: 0.4490 - accuracy: 0.9116

best_model_accuracy = lron_base_history.history['accuracy'][np.argmin(lron_base_history.history['loss'])]
best_model_accuracy

0.98570317029953

Observations

• Test Accuracy is 91%
• Training model accuracy with least loss is 99%.

Plotting the Confusion Matrix

# Complete the code to obtain the output probabilities
y_pred=lron_base_model.predict(X_test_normalized)

15/15 [==============================] - 1s 71ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)     # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()

Visualizing the prediction

#lron_da_model until I have another model

h, w, c = X_train_normalized[1000].shape
print('width:  ', w)
print('height: ', h)
print('channel:', c)


# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[2].reshape(1,h,w,c)))))   # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2])                                               # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[33].reshape(1,h,w,c)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33])                                              # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[59],)
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[59].reshape(1,h,w,c)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[59])                                              # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[36].reshape(1,h,w,c)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36])                                              # using inverse_transform() to get the output label from the output vector

width:   64
height:  64
channel: 3

1/1 [==============================] - 0s 37ms/step
Predicted Label ['Small-flowered Cranesbill']
True Label Small-flowered Cranesbill

1/1 [==============================] - 0s 34ms/step
Predicted Label ['Cleavers']
True Label Cleavers

1/1 [==============================] - 0s 33ms/step
Predicted Label ['Common Chickweed']
True Label Common Chickweed

1/1 [==============================] - 0s 24ms/step
Predicted Label ['Shepherds Purse']
True Label Shepherds Purse

Actionable Insights and Business Recommendations

# Plotting the classification report
cr=classification_report(y_test_arg,y_pred_arg,target_names=labels_plants)
print(cr)

                           precision    recall  f1-score   support

              Black-grass       0.70      0.54      0.61        26
                 Charlock       0.97      0.90      0.93        39
                 Cleavers       0.84      0.93      0.89        29
         Common Chickweed       0.97      0.97      0.97        61
             Common wheat       0.95      0.95      0.95        22
                  Fat Hen       0.98      0.98      0.98        48
         Loose Silky-bent       0.80      0.91      0.85        65
                    Maize       0.88      1.00      0.94        22
        Scentless Mayweed       0.93      0.96      0.94        52
          Shepherds Purse       0.90      0.78      0.84        23
Small-flowered Cranesbill       0.96      0.92      0.94        50
               Sugar beet       1.00      0.92      0.96        38

                 accuracy                           0.91       475
                macro avg       0.91      0.90      0.90       475
             weighted avg       0.91      0.91      0.91       475

Insights:

This model does a good job of classifying these plants, for the most part. Several of the models had difficulty with Black-grass and Loose Silky-bent, but this model seems to mostly have difficulty with the Black-grass. Precision is above 90% for many plants, meaning that for those plants we are very accurate.

Having more samples of the lower represented plant classes would likely help with improving classification, as would further epochs , though from experimentation, not by a lot. In other models not included here, it was seen that not resizing images also helps with identification, however, this takes a lot longer to train the models so the loss we get by reducing image size may offset that.

Overall, a 91% accuracy rating feels like a good result, and a suitable model for use.